This reverses an earlier change to support a feature that has been stripped
(for now). When said feature is put back in, it should support multi.
Setting this to allow mutlt = true allows for more confidence in tests.
Some refactoring to make sure errors get reported properly when things
go wrong. Wait until functions need to assert on the return value, and
this return value is more useful if it contains errors from nodes timing
out, Bad RPCs, etc.
Resolve failures with cuttlefish configuration changes in Riak 2.0.
Remove riak_control_upgrade, since riak_control should cover those use
cases completely.
The verify_busy_dist_port helper function cause_bdp:spam_nodes/1
recently changed to be more aggressive in triggering busy_dist_port
warnings. The function changed to spawn 1 million processes to ensure
the test generated enough activity to trigger the warnings, but that
number of processes exceeds the 256 thousand process limit that is the
Riak default. One consequence of this can be that the rex server
responsible for handling rpc calls can crash. In some cases this leads
to rpc calls by riak_test to shutdown the riak nodes involved in the
test to hang indefinitely. This change reduces the number of processes
spawned to 200 thousand. This should still be enough processes to
trigger the busy_dist_port warnings, but without exceeding the beam
process limit.
Allow intercept functions passed to rt_intercept:add/2 to be anonymous. In
compiled code they can either be a plain anonymous function, assuming they
don't use any variables from the surrounding context, or they can be a
2-tuple like this:
{[FreeVar1, ...],
fun(Arg1, ...) -> ... end}
where FreeVar1 etc. is a list of free variables to be closed over so that
they can be used within the anonymous function. For making interactive
calls to rt_intercept:add/2 from the Erlang shell, only the anonymous
function form is required, even if it uses free variables, though the
2-tuple form is also acceptable.
For compiled code, support for anonymous intercept functions is implemented
via a parse transform, and so to use anonymous functions the intercept
structure(s) containing them must be defined directly inline as part of the
final argument to rt_intercept:add/2, i.e., they cannot be first assigned
to a variable that is then used within the argument. This is because the
value of such a variable might not be visible to the parse transform.
Add a description of anonymous function intercepts to the README.
Prior to Riak 1.4.8 replication registers as a service prior to
completing all initialization tasks including establishing realtime
connections to sink clusters. This leads to a race condition in the
replication_upgrade and replication2_upgrade tests where the test may
begin writing data to the source cluster to verify the function of
realtime replication before the most recently upgraded node
establishes a connection to the sink cluster. The result of this is
that the data is silently discarded by the realtime replication system
and the test fails because all of the expected data is not replicated
and able to be read on the sink cluster. Change the
replication_upgrade and replication2_upgrade tests to explicitly wait
for the realtime connection to be established after each source
cluster node is upgraded before proceeding with the test.
Verify that upgrading Riak with Bitcask to 2.0 or later will trigger
an upgrade mechanism that will end up merging all existing bitcask
files. This is necessary so that old style tombstones are reaped,
which might otherwise stay around for a very long time. This version
writes tombstones that can be safely dropped during a merge. Bitcask
could resurrect old values easily when reaping tombstones during a
partial merge if a restart happened later.
Establish a new PB connection to the legacy node after it is upgraded
in order to avoid a failure. The PB connection may close if the node
upgrade takes too long and its reuse in such a case can lead to test
failure because use of the pid returns {error, disconnected} errors.
