The boost test framework has changed significantly from boost 1.34 to
1.37. Quite a few new features have been added, and some annoying bugs
have been fixed.
This change now builds the thrift tests against boost 1.37, and updates
them to use some of the newer features.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005162 13f79535-47bb-0310-9956-ffa450edef68
- TBufferedTransport::borrow() could block if not enough data was
available. Now it returns NULL immediately in this case, like all
other transports.
- TBufferedTransport::read() could block some data was available in the
readahead buffer, but not enough to satisfy the request. It would
attempt to call read() on the underlying transport, but this might
block. Now it just returns the remaining data in the readahead
buffer. The caller is responsible for calling read() again to get the
rest of the data they want.
- TFrameTransport::read() threw an exception if read() on the underlying
transport returned 0 when looking for a frame header. Now
TFrameTransport::read() returns 0, too. (It still throws an exception
if the underlying transport returns 0 after a partial frame or frame
header has been read.)
- TFDTransport::read() threw an exception on EINTR. Now it retries up
to 5 times, similarly to the way TSocket::read() behaves.
- TZlibTransport::read() could block when less data than was requested
is available. Now it only calls read() on the underlying transport
when it would otherwise have nothing to return.
This does mean that TZlibTransport::read() now often returns less data
than is actually available at the time. This required updating
several of the ZlibTest tests to use readAll() instead of read(),
since they previously assumed read() would return all available data.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005161 13f79535-47bb-0310-9956-ffa450edef68
Add tests that check to see whether or not read() and borrow() block
when called with a length larger than the amount of data currently
available.
At the moment, not all of the transports behave the same way. I believe
the desired behavior is:
When M bytes are available, and 0 < M < N:
- read(N): return M bytes immediately
- borrow(N): return NULL immediately
When 0 bytes are available:
- read(N): In this case, it is acceptable either to immediately return
0, or to block until some data is available. If the transport
blocks, it returns immediately when some date becomes available,
even if less than N bytes are available.
- borrow(N): return NULL immediately
- The borrow() tests fail when using TBufferedTransport.
TBufferedTransport incorrectly blocks until the amount of data
requested is available.
- test_read_none_available() fails when using TFramedTransport.
Calling read() on a TFramedTransport when no data is available throws
an exception instead of returning 0.
- test_read_none_available() fails when using TFDTransport. This is
partly just an artifact of the fact that I use SIGALRM as part of this
test. Unlike TSocket, TFDTransport doesn't retry after EINTR.
- test_read_part_available() fails when using TZlibTransport around a
transport that has blocking read() behavior. TZlibTransport::read()
loops calling read() on the underlying transport. It should probably
break out of the loop and return to the caller as soon as it has
uncompressed any data, even if it is less than requested and more
might be available. Once some data has been uncompressed,
TZlibTransport cannot risk calling read() again since it might block.
Will commit fixes for these separately.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005160 13f79535-47bb-0310-9956-ffa450edef68
UpdateS TransportTest so that the wrapper transports
(TBufferedTransport, TFramedTransport, TZlibTransport) are tested with a
wider variety of inner transports. Previously they were only tested
using TMemoryBuffer. Now all other transports are also tested wrapped
inside each of these transports.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005159 13f79535-47bb-0310-9956-ffa450edef68
Reduce the default test buffer sizes by about 30x, reducing the time it
takes to run TransportTest from about 1 minute to about 2 seconds. I
added a --size-multiplier argument that can be used to adjust the sizes
of all test buffers, so developers can still run with large buffer sizes
when desired.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005157 13f79535-47bb-0310-9956-ffa450edef68
Previously flush() had race conditions that could cause it to return
before all data had actually been flushed to disk. Now the writer
makes sure both buffer queues have been flushed when forceFlush_ is set.
Also, flush() did not wake up the writer thread, so it normally had to
wait for the writer thread to wake up on its own time. (By default,
this could take up to 3 seconds.)
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005156 13f79535-47bb-0310-9956-ffa450edef68
Several of the TFileTransportTest tests check wall clock time to make
sure the writer thread processes operations quickly enough, and isn't
hanging. However, this can easily result in false failures if we don't
get enough processor time. This commit makes a few changes to reduce
the number of these failures.
- No longer fail if a single destructor call takes more than 500us.
We only require 90% of the calls to complete in 500us. No call may
take more than 100ms, though. With this change, the test passes most
of the time now, even while an "fbmake opt" task is running in
parallel.
- In the flush_max_us tests, make sure the writer thread is started
before we start recording timing. Otherwise, creating the thread
could take long enough to throw off the numbers for the first fsync()
call.
Also tested with the pthread_cond_signal() in the TFileTransport
destructor commented out, to make sure the test still fails properly
when the destructor takes too long.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005155 13f79535-47bb-0310-9956-ffa450edef68
Set closing_ to true before we wake up the writer thread in the
destructor. This way the writer thread flushes the data and exits when
it is woken up.
Previously the writer thread could end up going through 2 full timeout
cycles before exiting. The writer thread could wake up, notice it has
nothing to do since closing_ is not set, and immediately go back to
sleep. The destructor would then proceed to call flush(), which would
wait for the writer thread to wake up (1 full time out) and clear the
forceFlush_ flag. After flush returns, the destructor would set
closing_. It could take the writer thread another full timeout to wake
up again and process this flag.
There were also some points where the worker threads would detect the
closing_ state in their loops and automatically close the file, but did
not zero out the file descriptor, then the destructer attempted to close
the same file. Fix by simply zeroing out the fd_ at these points.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005154 13f79535-47bb-0310-9956-ffa450edef68
Now that TZlibTransport::flush() behaves the same way as other
transports, there is no need to distinguish between RPC and standalone
behavior for TZlibTransport.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005152 13f79535-47bb-0310-9956-ffa450edef68
Previously, TZlibTransport::flush() finished the zlib stream, so calling
write() after flush() would result in an error. Now it just flushes the
data, without finishing the stream. A new TZlibTransport::finish()
function has been added to finish the stream.
This breaks compatibility. I'm aware of anyone using this code outside
of Facebook, though.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005151 13f79535-47bb-0310-9956-ffa450edef68
This way we no longer have to have a huge hard-coded list of numbers in
the source code. The distribution is randomly generated for each run.
(Although the --seed argument can be used for repeatablity.)
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005150 13f79535-47bb-0310-9956-ffa450edef68
Previously, ZlibTest read a file from disk to get data to test with.
It would fail unless gen-cpp/DebugProtoTest_types.tcc was present in the
current directory and was at least 32kB long.
Now ZlibTest simply generates 3 separate buffers to test with. The
first buffer is just all "a"s, the second is some random sequential
runs, and the third is completely random. They usually seem to have
compression ratios of around 315:1, 4:1, and 1:1, respectively.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005148 13f79535-47bb-0310-9956-ffa450edef68
Tests various transport types by writing data, and verifying it can be
read back successfully. Tests both virtual calls (accessed as
TTransport*) and non-virtual calls (accessed as the appropriate pointer
type, or as TBufferBase* when appropriate).
This is similar to some of the tests already performed in
TMemoryBufferTest and ZlibTest.cpp. However, this tests a few more
transport types, and it interleaves read and write calls more heavily.
(It currently exposes some bugs in flush() in a couple transports.) It
also exercises both the virtual and non-virtual code paths, now that
read() and write() only result in virtual calls when invoked on a
TTransport*.
TFileTransport currently has several race condition bugs, so most of the
TFileTransport tests ususally fail. It also has some performance bugs,
causing the TFileTransport tests to take a long time. Will fix those
issues separately.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005146 13f79535-47bb-0310-9956-ffa450edef68
Convert TBinaryProtocol and TCompactProtocol to template classes, taking
the transport class as a template parameter. This allows them to make
non-virtual calls when using the template, improving serialization
performance.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@1005136 13f79535-47bb-0310-9956-ffa450edef68
