Riak 1.1.2 RC1 has been released

Release candidate for Riak 1.1.2 on the download page.

For those of you eager to try out some of the bug fixes, please head on over there, download it and let us know what you find! Download Page: http://basho.com/resources/downloads/ The release notes for the 1.1.2 release as well as the entire 1.1.x series can be found here:

https://github.com/basho/riak/blob/1.1/RELEASE-NOTES.org

The bugs fixed for 1.1.2 are:
  • Work-around for net_kernel:get_status() race condition Handoff reported as started even
  • if rejected for exceeding handoff concurrency
  • Intermittent hang with handoff sender
  • handoff of fallback data can be delayed or never performed
  • Chicken & egg problem with vnode proxy process start & registration?
  • Put FSM crashes if forwarded node goes down

Riak 1.1.1 RC 1 released

Basho has released the 1.1.1 RC 1 version of Riak which can be downloaded here

MapReduce Changes

This release includes fixes for systems with heavy MapReduce load.

Pipe maintains a limit on the number of active worker processes in the system. Workers are created on demand when the pipe executes – they are not reserved at pipe creation time.

Prior to 1.1.1 not all fittings (e.g. get object from riak, map value, reduce) were checking to make sure their output was being delivered to the next stage of the pipeline. This has been changed so that the pipe will error if processing errors or system limits are hit.

As a result of the change if the pipe is overloaded clients will see errors of this form:{error,<<"Error sending inputs: {error,worker_limit_reached}">>}

Clients can either decide to retry (preferably with a backoff/retries limit mechanism) or if the hardware has sufficient capacity the number of workers per vnode can be increased by setting worker_limit in etc/app.config.

{riak_pipe, [{worker_limit, #Workers}]}

Different numbers of workers are used based on the type of MapReduce query so tuning will be very workload dependent.

If you are using Javascript MapReduce you will also need to increase the number of Javascript virtual machines in the riak_kv section ofapp.config.

            {map_js_vm_count,  #MapVMs},
            {reduce_js_vm_count, #ReduceVMs}

Again, tuning will be workload dependent. If sufficient resources are available setting #MapVMs to the maximum number of JS MapReduce jobs being run in parallel will give best results and #ReduceVMs set to number of jobs / number of nodes plus a margin for overload.

Bugs Fixed

  • Map-reduce jobs fail during rolling upgrade to 1.1
  • Race Condition causes Javascript VMs to never be returned to VM Pool
  • Avoid {case_clause, ok} errors when vnode backend fails to start

Riak 1.1 has been released

Riak 1.1 has been released, it introduces both our new administration console Riak Control and our new diagnostic package Riaknostic.   Most importantly to some of you though, is its much improved behavior in large clusters.

Others improvements includes:

Bitcask Improvements

  • CRC checks have been added to hintfiles for bitcask

LevelDB Improvements

  • Snappy (compression algorithm from Google) has been enabled
  • The compaction algorithm has been tuned to reduce delays due to compaction

Other backend changes

  • Multi-backend now supports 2i properly

Lager Improvements

  • Tracing support (see the README)
  • Term printing is ~4x faster and much more correct (compared to io:format)
  • Bitstring printing support was added

MapReduce Improvements

  • The MapReduce interface now supports requests with empty queries. This allows the 2i, list-keys, and search inputs to return matching keys to clients without needing to include a reduce_identity query phase.
  • MapReduce error messages have been improved. Most error cases should now return helpful information all the way to the client, while also producing less spam in Riak’s logs.

Download: http://basho.com/resources/downloads/

Realease notes: https://github.com/basho/riak/blob/master/RELEASE-NOTES.org

 

Riak 1.1RC1 has been released

Jared Morrow jared at basho.com announce the availability of Riak 1.1.0 RC (release candidate) 1.  Planning the 1.1 release to be a shorter cycle from first RC to final release (barring any major event or bug that is found with RC1 obviously).

Download from the downloads page http://basho.com/resources/downloads/ under 1.1 or CURRENT.

An extremely rough initial draft of the 1.1 release notes is available here:  https://github.com/basho/riak/blob/master/RELEASE-NOTES.org

 

Major Features and Improvements for Riak:

  • Riak Control
  • Riaknostic
  • Bitcask Improvements
  • Lager Improvements
  • Bitstring printing support
  • LevelDB Improvements
  • Snappy Support Enabled
  • Compaction Changes
  • Riak KV Improvements
  • Log post-commit errors
  • Listkeys Backpressure

 


Riak Compared to Neo4j

Basho, the co,pamy behing Riak, has just published a nice paper comparing its solution to Neo4j, even if both solution are meant for storing fundamentally different types of data:

  • Riak is a document database and key/value store, designed to store semi-structured documents or objects of varying sizes.
  • Neo4j is a graph database, designed to store and traverse a network of related information. (e.g. Social Networks)

In most cases, the needs of your application will clearly dictate whether you should use a key/value store or a graph database. And in many cases, it may make sense to combine the two. An application like Facebook, for example, might store user profile information, wall posts, and images in a key/value or document database, and store the network of friends and associations in a graph database.

Read the entire article on Basho’s wiki:

http://wiki.basho.com/Riak-Compared-to-Neo4j.html

 

Riak 1.0.3 has been released

Riak 1.0.3 has been released and can be downloaded on official Basho‘s website http://downloads.basho.com/riak/riak-1.0.3/

 

Bugs Fixed

 

Additions

riak-admin top

riak-admin can now run top against the server. Top provides information about what the erlang processes inside of Riak are doing. Top reports process reductions (an indicator of CPU utilization), memory used and message queue sizes.

Start with riak-admin top and exit with double-^C

busy_port / busy_dist_port

Riak will now log whenever distributed erlang is overloaded by logging busy_dist_port messages on the console and to the logs. If the message is being logged frequently, the buffer used for distributed erlang can be increased by adding +zdbbl KBytes in the vm.args file (in /etc/riak or /opt/riak/etc depending on platform).

Bitcask Support for O_SYNC

Bitcask has been reworked to perform I/O directly on the Erlang scheduler threads; this is the result of extensive load testing in a production environment. As a result of this change, it can now support O_SYNC on Linux.

New Claim Algorithm

The new claim algorithm introduced in the 1.0 release has been set as the default for 1.0.3.

The new claim algorithm significantly reduces the amount of ownerhip shuffling for clusters with more than N+2 nodes in them. Changes to the cluster membership code in 1.0 made the original claim algorithm fall back to just reordering nodes in sequence in more cases than it used to, causing massive handoff activity on larger clusters. Mixed clusters including pre-1.0 nodes will still use the original algorithm until all nodes are upgraded.

Changes to Handoff Concurrency

The default value for handoff_concurrency was set to 1 to err on the side of caution and not overload the cluster with handoff traffic.

 

Main security tools NMAP integrating detection for NoSQL solution

Nmap (“Network Mapper”) is a free and open source utility for network exploration or security auditing. Many systems and network administrators also find it useful for tasks such as network inventory. Its also used to determine what hosts are available on the network, what services (application name and version) those hosts are offering.

Since the 2nd of January 2012, scripts for NMAP are available to handle few NoSQL solutions:

http://seclists.org/nmap-dev/2012/q1/11

So using NMAP and those latest scripts you’ll be able to detect on your network the server running riak, memcached or redis as well as the installed version.

 

 

 

Basho lands new funding with an additional $5m

Basho, the company behind the great NoSQL data solutions Riak has closed an additional equity round of funding of $5m. The round included only existing investors – and the funds will be used to make Riak an even better product.

In addition, Basho and its partner Trifork have helped the National Board of E-Health in Denmark create a scalable, highly available database for its Shared Medicine Card program. This system in time will enable all Danish citizens and their physicians to access their complete prescription history from any device in real time.

Congratulations to http://basho.com/

 

Attacking and Defending NoSQL

One of the first  paper on NoSQL security. Its  a great presentation  by Bryan Sullivan at the RSA conference, it introduces the main security issue of the NoSQL solutions such as:

  • NoSQL injection, just like SQL injection but manipulating the JSON string instead of the SQL query
  • Authentication is unsupported or discouraged  within NoSQL solution which is a big issue when combined with REST API
  • SSJS Injection aka Server-side JavaScript injection

 

Basho released Riak 1.0

Riak 1.0 is officially released,  its  a Dynamo-inspired database that is being used in production by companies like Mozilla and Comcast. Riak scales predictably and easily and simplifies development by giving users the ability to quickly prototype, test, and deploy their applications.

A truly fault-tolerant system, Riak has no single point of failure. No machine is special or central in Riak, so developers and operations professionals can decide exactly how fault-tolerant they want and need their applications to be.