- Clean up lib/cpp/Makefile.am to use AM_CXXFLAGS etc instead of
target-specific so we can take advantage of default flags.
- Use top_builddir in a few places instead of top_srcdir.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@666489 13f79535-47bb-0310-9956-ffa450edef68
On FreeBSD (and possibly other BSDs), binding to ipv6 is restricted unless the
sysctl net.inet6.ip6.v6only=0 or the socket option IPV6_V6ONLY is turned off.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@666367 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Add an argument to the Mutex constructor.
It takes a pointer to a function that initializes the internal pthread_mutex_t.
We provide initializers for default pthread_mutex_t (which is our default),
adaptive mutexes, and recursive mutexes.
Reviewed By: hzhao, psaab, mcslee
Test Plan: Built libthrift.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@666362 13f79535-47bb-0310-9956-ffa450edef68
Adds an #ifdef'd block that calls ProfilerRegisterThread() if
"REGISTER_THREAD_GOOGLE_PERFTOOLS" is set. Apparently, this makes the
Google perftools CPU profiler work much better on x86-64.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@666358 13f79535-47bb-0310-9956-ffa450edef68
The old implementations of the memory-based transports (TBufferedTransport,
TFramedTransport, and TMemoryBuffer) shared very little code and all worked
a bit differently. This change unifies them all as subclasses of a single
base (TBufferBase) which handles the fast-path operations (when requests
can be satisfied by the buffer) with inline methods (that will eventually
be made nonvirtual in the template branch) and calls out to pure-virutal
methods to handle full/empty buffers. All of the buffer-management is now
done in terms of "base and bound" pointers rather than "pos" integers.
These classes were moved to TBufferTransports.{h,cpp}. The .h is included
in TTransportUtils for backwards compatibility.
Also added a "TShortReadTransport" to assist testing transports.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665676 13f79535-47bb-0310-9956-ffa450edef68
Summary:
These methods previously took name as a const string&. While that way
is more idiomatic, it requires a temporary string to be constructed
when we pass a string literal (which is always). This was significantly
slowing down the serialization of field-heavy structures. This change
will break ABI compatibility, but the serialization speed boost with
no external API changes is too important to pass up.
Reviewed By: mcslee, aditya
Test Plan: make check
TracCamp Project: Thrift
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665672 13f79535-47bb-0310-9956-ffa450edef68
calling decrement with -1 even though param is an integer. Just made it clean by adding a decrement function
reviewed by: mcslee
revertable
test plan: ran on a test server
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665669 13f79535-47bb-0310-9956-ffa450edef68
Summary: Added support for persistent conns in TSocketPool
Also, added some util functions in TNonblockingServer
Reviewed By: mcslee
Test Plan: Ran a test search cluster with these changes - open was only called once (I put fprintfs in open and close), after which the socket was reused
Revert: OK
TracCamp Project: Thrift
DiffCamp Revision: 11425
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665668 13f79535-47bb-0310-9956-ffa450edef68
Summary: TServerSocket uses a socket pair to signal interruption to the select() call on its main server socket. A bug in the code prevented the select() from listening for events on the internal socket pair; in fact, it was accidentally listening to file descriptor 1.
Reviewed By: aditya
Test Plan: Use a server that supports a call to shutdown, which should signal an interrupt to the internal socket, and should result in an immediate and clean termination of the server socket thread.
Revert: OK
TracCamp Project: Thrift
DiffCamp Revision: 11932
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665658 13f79535-47bb-0310-9956-ffa450edef68
- Add some missing namespace qualifiers that are not used in the glibc case.
- Move an include from Thrift.h to Thrift.cpp.
- Include boost/lexical_cast.hpp in TJSONProtocol, in case we ever eliminate
it from our header files.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665656 13f79535-47bb-0310-9956-ffa450edef68
Summary: - select has a restricted max fd set size. While this should not be an issue in the normal case for
TServerSocket because it is started when the process starts, it could be a problem if someone wanted
to start a new server socket at a later point in time when socket values could be greater than max.
Reviewed By: dreiss
Test Plan: - Compiled.
- Deployed search tier and made sure it could serve queries.
Revert: OK
TracCamp Project: Thrift
DiffCamp Revision: 11076
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665649 13f79535-47bb-0310-9956-ffa450edef68
Summary: - Move strerror_s to Thrift.h (was previously in TTransportException.h)
- Capture errno as soon as syscall returns failure and make it part of error message.
- Cleaned up several instances of the wrong error value being printed.
- More consistently pass the errno in the TTransport Exception
- Add more consistent error logging for the various transport failure modes
Reviewed By: dreiss
Test Plan: - compile everything.
- test on search tier
Revert: OK
TracCamp Project: Thrift
DiffCamp Revision: 11077
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665648 13f79535-47bb-0310-9956-ffa450edef68
Summary: fd_set, which we pass into select only supports fds with values < 1024.
fds can be >= 1024.
this can cause memory corruption
learning this stuff over 2 weeks of examining crazy core dumps is way more fun that reading it up somewhere.
poll takes an array of fds insetad, so your fd can be any int
Reviewed By: dreiss
Test Plan: ran inside a test search cluster
Revert: OK
DiffCamp Revision: 10731
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665647 13f79535-47bb-0310-9956-ffa450edef68
Previously, TPipedTransport wrote it's full buffer to the "pipe" and
fully reset its buffer on a readEnd. This assumed that the buffer was
fully read at that point. This is not the case if requests are pipelined.
This change makes it only pipe out the portion that has been read and
copy the unread portion to the beginning of the buffer.
Also add a test that verifies the new functionality.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665646 13f79535-47bb-0310-9956-ffa450edef68
Summary:
realloc doesn't free the original pointer when it returns null.
Reviewed By: cpiro
Test Plan: make check
TracCamp Project: Thrift
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665626 13f79535-47bb-0310-9956-ffa450edef68
Summary: 1] Added empty constructor
2] Added function to set server list
3] Made getServers return by ref
4] Made list of servers be a list of pointers
The last change is so that someone can pass in the list of servers and rely on TSocketPool to change the failure time directly on the TSocketServer object
Reviewed By: aditya
Test Plan: Ran with a test search server
Revert: OK
DiffCamp Revision: 9529
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665571 13f79535-47bb-0310-9956-ffa450edef68
Summary: This adds the discussed interface to TMemoryBuffer, as follows:
- writePtr(size) returns a ptr you can write (size) bytes to
- wroteBytes() lets it know you wrote that many bytes
To do this, I refactored an:
- ensureCanWrite(size) private func
Reviewed By: dreiss
Test Plan: works in my test environment
Revert: OK
TracCamp Project: Thrift
DiffCamp Revision: 8739
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665555 13f79535-47bb-0310-9956-ffa450edef68
Summary: - Added 'struct timeval' toMilliseconds and toTimeval converters modeled on the timespec ones
- Added constants for microsecs, and got rid of the 500000 and 500 magic #'s :)
Reviewed By: dreiss
Test Plan: works in my test env
Revert: OK
DiffCamp Revision: 8742
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665554 13f79535-47bb-0310-9956-ffa450edef68
- Adding ability to use a vector of TSocketPoolServers to construct a TSocketPool
- Ability to get back the list of TSocketPoolServers
This is especially useful in multithreaded client code that
will just keep around the list of servers, and create the pool
on every request. Since TSocketPool updates the failure stuff,
we need a way to get back the updated TSocketPoolServers
Reviewed By: aditya
Test Plan: just compiling the code
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665537 13f79535-47bb-0310-9956-ffa450edef68
Wasn't setting the host or port for each server in the pool on open.
Reviewed By: dbraginsky
Test Plan: Just compiled. Will test with dbraginsky's server.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665535 13f79535-47bb-0310-9956-ffa450edef68
Summary: Replicating php logic: If opening fails enough times, mark server as down for some amount of time
Reviewed By: aditya
Test Plan: compiling thrift - any good test ideas?
Revert: OK
DiffCamp Revision: 8381
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665534 13f79535-47bb-0310-9956-ffa450edef68
Previously, the Base64 decoding table only covered ASCII characters,
but the decoding function looked up any character in the table.
This change extends the table.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665532 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Added a LookaheadReader to the TJSONProtocol so it doesn't have to
rely on the transport to borrow.
Also added a check to a corner case and fixed up some comments and whitespace.
Reviewed By: mcslee
Test Plan: make check
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665491 13f79535-47bb-0310-9956-ffa450edef68
Summary:
There were a few places where we were calling malloc/reallaoc/free without
including cstdlib (or stdlib.h). This is broken, but it worked because
other headers that we were including included stdlib.h. However, on a
platform where this wasn't true, it broke the Thrift build. This change
adds the proper includes. It also changes malloc to std::malloc (same
with realloc and free) in a few places, because that is the correct way
of doing it when you include cstdlib.
Reviewed By: mcslee
Test Plan: Compiled Thrift.
Revert Plan: ok
Other Notes:
This bug was noticed by a Thrudb user, and the patch was sent in by
Ross McFarland.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665487 13f79535-47bb-0310-9956-ffa450edef68
Summary:
This change adds a new and exciting protocol to Thrift. It uses
RFC-compliant JSON as the wire protocol and is fully human readable.
(once a little whitespace has been inserted.) Unlike the existing
JSON protocol for Java, which is intended to allow Thrift data to be
transferred to scripting languages, this protocol is lossless and fully
read-write. It was written by Chad Walters of Powerset and reviewed
by David Reiss.
Tested by running make check.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665482 13f79535-47bb-0310-9956-ffa450edef68
Summary:
The upcoming TJSONProtocol handles string and binary types quite differently.
This change makes that distinction in all parts of the C++ binding.
Java already distinguished between string and binary, but this change
also updates the Java skip method to skip over strings as binary
so we don't get encoding errors when skipping binary data.
Reviewed By: mcslee
Test Plan: make check
Revert Plan: ok
Other Notes:
I just pulled this out of Chad Walters' JSON patch.
The only other change was adding readBinary (or was it writeBinary)
to TDenseProtocol. Maybe inheriting from TBinaryProtocol wasn't a good idea.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665481 13f79535-47bb-0310-9956-ffa450edef68
Summary: Use comment trick in params that are unused to prevent warnings
Reviewed By: dreiss
Test Plan: Generate C++ code and compile -W -Wall
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665469 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- Expanded tabs to spaces where spaces were the norm.
- Deleted almost all trailing whitespace.
- Added newlines to the ends of a few files.
- Ran dos2unix on one file or two.
Reviewed By: mcslee
Test Plan: git diff -b
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665467 13f79535-47bb-0310-9956-ffa450edef68
Summary:
There were some weird cases where the implicit conversion from
const char* to std::string was causing the wrong constructor to be called.
There wasn't really a clean workaround, so we're dropping the string
constructors.
Reviewed By: mcslee
Test Plan:
Ran the test.
Grepped around the /projects tree for uses that had to fixed,
and fixed them.
Revert Plan: ok
Other Notes:
This risk was pointed out by Ben Maurer.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665461 13f79535-47bb-0310-9956-ffa450edef68
Summary:
This didn't get updated when the interface changed,
and the cast silenced the warning message. Stupid C.
Reviewed By: mcslee
Test Plan: Trust Chad Walters.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665457 13f79535-47bb-0310-9956-ffa450edef68
Chad Walters is writing a JSON protocol for Thrift, but he wants
borrow to always succeed. That would be a pain to implement,
but here is a first step: borrow will almost always work with
TBufferedTransport.
Reviewed by: mcslee
Test Plan: Ran the DenseProtocol test and Zlib test, but more needs to be done.
Other Notes:
Also reviewed by Chad Walters, and maybe Ben Maurer.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665455 13f79535-47bb-0310-9956-ffa450edef68
Summary:
I don't know what I was thinking when I first wrote this.
It makes sense that the transport might not want to allocate its own memory,
so the protocol is expected to provide a buffer for the data.
However, if the transport already has the data buffered,
there is no need to memcpy it; it can just return a pointer into its buffer.
The new interface still requires the protocol to provide a buffer,
but allows the transport to return a pointer to an interal buffer.
In addition, I made len a pass-by-pointer parameter so that
the transport can return more than the requested data if it has it
available in its buffers.
Reviewed By: mcslee
Test Plan: Ran the DenseProtocol test and the Zlib test.
Revert Plan: ok
Other Notes:
Also got this reviewed by Chad Walters from Powerset.
Ben Maurer suggested making len a reference parameter.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665454 13f79535-47bb-0310-9956-ffa450edef68
the threadpoolserver should just crash if an unknown exception type is thrown.
This should probably be true for all server types, will put this on the TODO list.
reviewed by jwang
test plan: network selector servers in production
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665412 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Ben Maurer suggested that it would make sense for Thrift to build as
a single project, with one configure.ac and multiple Makefile.am.
He was also kind enough to do the heavy lifting, and this commit
is the application of his patch (with minor modifications).
The most significant visible change from this diff is that
in order to buidl one of the thrift sub-projects (i.e.: the compiler,
the C++ library, or the Python library) you must run bootstrap.sh
and configure in the Thrift root, then make in the specific project.
Users who want to build and install the Python library but
can't run configure because they don't have Boost can simply
run setup.py directly.
Reviewed By: mcslee
Test Plan: Built Thrift from scratch.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665409 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- it should be possible to specify a zero timeout if the pendingTaskQueue
is already full
Reviewed By: marc
Test Plan:
- Tested in search
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665406 13f79535-47bb-0310-9956-ffa450edef68
Summary: const so that catching via "catch (const TTransportException& e)" works
properly, and throw() for good convention.
Reviewed By: mcslee
Test Plan: still compiles
Revert: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665403 13f79535-47bb-0310-9956-ffa450edef68
Reviewed By: boz
Test Plan:
dreiss@dreiss-vmware:safe_concurrency:thrift/test$ nl -ba guard_test.cc
1 #include "../lib/cpp/src/concurrency/Mutex.h"
2 using facebook::thrift::concurrency::Guard;
3 void test() {
4 facebook::thrift::concurrency::Mutex m;
5 Guard(m);
6 Guard g1(m);
7
8 facebook::thrift::concurrency::ReadWriteMutex rwm;
9 facebook::thrift::concurrency::RWGuard(m);
10 facebook::thrift::concurrency::RWGuard g2(rwm);
11 }
dreiss@dreiss-vmware:safe_concurrency:thrift/test$ g++ -c guard_test.cc
guard_test.cc: In function ‘void test()’:
guard_test.cc:5: error: ‘incorrect_use_of_Guard’ was not declared in this scope
guard_test.cc:9: error: ‘incorrect_use_of_RWGuard’ is not a member of ‘facebook::thrift::concurrency’
dreiss@dreiss-vmware:safe_concurrency:thrift/test$
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665402 13f79535-47bb-0310-9956-ffa450edef68
Summary: O_APPEND means that even if the file pointer is in the middle of the file, the next write will go to the end of the file. So when initializing the TThriftTransport to be written to, we need to truncate the file after the last good event. I'm pretty sure this was what was causing queuemap log corruption when the multiplexer is restarted.
The corruption doesn't necessarily go away at the next chunk boundary because of the offset being recalculated. I think it's good to have the offset recalculated, but then we shouldn't use the old chunk boundary data there, but rather use the newly calculated offset to determine the padding (could be an entire chunk, but that's okay -- way better than corruption)
Reviewed By: mcslee
Test Plan: Wrote to a ThriftFile using local scribe instance. Printed it. Then cat'd garbage to it from the command line. Then wrote to it again. Then printed it again. Seems to work fine, whereas with old scribe binary the later entries were corrupted.
Revert: OK
DiffCamp Revision: 4683
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665388 13f79535-47bb-0310-9956-ffa450edef68
Summary: "Default TApplicationException" is pretty meaningless; if we have this
extra type information, we might as well present it.
Reviewed By: mcslee
Test Plan: caught a couple of INVALID_MESSAGE_TYPE tapplicationexceptions
Revert: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665383 13f79535-47bb-0310-9956-ffa450edef68
Summary: Such that users can supply an event handler to a server that will be used to signal various events that take place inside the server core.
Reviewed By: dreiss
Test Plan: Rebuilt all servers, work by default
Other Notes: Partially submitted and also reviewed by Dave Simpson at Powerset
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665371 13f79535-47bb-0310-9956-ffa450edef68
Summary: Need to pass hints including AI_ADDRCONFIG to getaddrinfo to make sure no IPv6 addresses come back on non-IPv6 enabled systems
Reviewed By: mcslee
Other Notes: Submitted by Paul Saab
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665368 13f79535-47bb-0310-9956-ffa450edef68
Summary: This allows the event loop to be shared across different components of a program of for a separate thread in a TNonblockingServer to safely use its own libevent code without conflicts.
Reviewed By: mcslee
Test Plan: Updated test/ committed here
Other Notes: submitted by Ben Maurer, patched in by mcslee with slight modifications
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665364 13f79535-47bb-0310-9956-ffa450edef68
Summary: Need to use IPv6 compliant constructs here. Submitted by Paul Saab.
Reviewed By: dreiss
Test Plan: Rebuild Thrift TNonblockingServer with this patch and verify behaves just the same
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665353 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- unflushed data should be flushed when transport is closed
- this diff calls flush for TBufferedTransport and TFramedTransport
Reviewed By: dreiss
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665323 13f79535-47bb-0310-9956-ffa450edef68
Summary: "Porting" this feature from PHP; I find it useful in the Synapse client.
Reviewed By: mcslee
Test Plan: It compiles, ship it!
Revert: svn
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665304 13f79535-47bb-0310-9956-ffa450edef68
Summary: Same thing as the previous PHP change. Also includes a new constructor for easy building of a TSocketPool with a single host (for later filling in via addServer) without extra std::vector boxing/unboxing.
Reviewed By: mcslee
Test Plan: Synapse c++ client at r62896 uses this.
Revert: OK
TracCamp Project: Thrift
DiffCamp Revision: 909
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665297 13f79535-47bb-0310-9956-ffa450edef68
Summary:
just some simple styling changes. moving using boost::shared_ptr to .cpp
and using the full path name in .h
Reviewed by: boz, dreiss
Test Plan: compiled and tested on the adfinder
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665289 13f79535-47bb-0310-9956-ffa450edef68
Summary: we need to use a shared_ptr instead of an old fashioned one if we're
going to stick with this PIMPL model
Reviewed By: dreiss, marc
Test Plan: test program didn't fail or leak memory, foreman (fb303 client)
worked without problem
Revert Plan: just make sure you find some other solution to this problem
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665280 13f79535-47bb-0310-9956-ffa450edef68
Summary: also added myself to CONTRIBUTORS.
Reviewed By: marc
Test Plan: the following program no longer leaks memory (valgrind):
int main(int argc, char **argv){
Mutex mu;
mu.lock();
mu.unlock();
}
Revert Plan: ok
Notes: this is kind of important
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665279 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- make dist is no fun. Had to add a lot of stuff to Makefile.am to
make sure stuff got included in the tarball.
- Added a permanent NEWS file with the release notes for this release.
- clean.sh removes ylwrap also.
- Gave it a real version for make dist.
- Added ACLOCAL_AMFLAGS to lib/cpp/Makefile.am to make it easier to
work on configure.ac.
- Made concurrency_test a non-installed binary because
no one wants to install it.
- Dropped some symlinks that were annoying make dist.
- Updated README.
Test Plan:
./bootstrap.sh && ./configure && make dist
Compared contents of tarball to source tree.
Did this from a fresh git tree so no gross untracked files would be around.
On devrs: ./bootstrap.sh && ./configure && make install DESTDIR=/tmp/relinst1
Made sure concurrency_test was built but not installed.
Checked zlib URL with firefox.
Reviewed By: mcslee
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665274 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Same as the last (reverted) zlib patch,
but this time with way more awesome support for building
with no zlib headers installed.
Reviewed By: mcslee
Test Plan:
- Did lots of really pathological stuff in my VMware.
- On devrs004:
./bootstrap.sh && ./configure && make && make install DESTDIR=/tmp/tzinst && echo "Yay"
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665269 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- Totally re-did the libevent check.
- Thrift now compiles and instals cleanly without libevent.
- (libevent is still needed to build libthriftnb.)
Reviewed By: mcslee
Test Plan:
- Ran configure and make in various configurations in my VMware.
- Clean build and install of Thrift on devrs004 (into /tmp).
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665267 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Add a new transport to the C++ library: TZLibTransport.
This wraps another transport (just like TBufferedTransport),
compresses the data written to it, and uncompresses the data it reads.
Reviewed By: mcslee
Test Plan:
For ax_lib_zlib.m4
- Ran ./bootstrap.sh.
- Looked at configure.
- Ran ./configure.
- Looked at config.status
- Ran ./configure --with-zlib=/usr.
- Looked at config.status
- Changed configure.ac to AC_LIB_ZLIB(1.2.4)
- Ran ./configure.
- Watched it fail.
For TZlibTransport
- test/ZlibTest.cpp
- Code coverage (hotness) report sent with review.
- 100% line coverage for all the good stuff.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665262 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- TDenseProtocol now includes a part of the struct fingerprint in
the serialized message, to protect from unserialzing trash.
- A lot of cleanups and commenting for TDenseProtocol.
- A lot of test cases for same.
Reviewed By: mcslee
Test Plan: test/DenseProtoTest.cpp
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665257 13f79535-47bb-0310-9956-ffa450edef68
Reviewed By: mcslee
Test Plan:
test/DenseProtoTest.cpp
Still have to test:
- Bounds checking.
- borrow/consume on TBuffered and TFramed.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665252 13f79535-47bb-0310-9956-ffa450edef68
Summary:
To support the upcoming variable-length encoding of integers in
TDenseProtocol, augment Thrift transports with two new methods:
borrow and consume. Borrow copies some data out of the transport
without consuming it, and consume... consumes it.
Reviewed By: mcslee
Test Plan:
Thrift compiles.
Should check in variable-length integer code for TDenseProtocol
right after this (git ruulz).
Revert Plan: revert stuff that uses these methods also.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665251 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- Made some stuff in TBinaryProtocol protected instead of private.
- Added a preliminary version of TDenseProtocol. This is still
super highly experimental and gross, and I wrote a super scary
comment to explain that to anyone foolish enough to try to use
this in its current state.
Reviewed By: mcslee
Test Plan: test/DenseProtoTest.cpp
Revert Plan: ok
Memcache Impact:
Save memory if/when people use it.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665247 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Local reflection typespecs for structs now have a dummy T_STOP field at the end
so we don't have to check the size on every iteration.
They also contain information about which fields are optional.
Also put a static pointer to the reflection in each structure.
Reviewed By: mcslee
Test Plan: test/DenseLinkingTest.thrift
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665246 13f79535-47bb-0310-9956-ffa450edef68
Also, checking in changes to the php generation code I made awhile ago, but forgot to check in
Reviewed By: aditya, mcslee
Test Plan: Compiled thrift and falcon, and confirmed that falcon's print binary still worked
Revert Plan: revertible
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665245 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- this was causing some bizarre issues with seeking to a particular point in the log
- I can't believe we haven't run into this before
Reviewed By: jwang
Test Plan: replayed log file from a partiicular point
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665242 13f79535-47bb-0310-9956-ffa450edef68
Summary:
The compiler now takes a "-dense" flag that will cause it to
generate some extra metadata for C++. This metadata will be used by
TDenseProtocol. This should be the last compiler change necessary
to enable that feature.
Reviewed By: mcslee
Test Plan: test/DenseLinkingTest.thrift
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665240 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- Give reflection_limited a nice clean Python module.
- if/regen.sh now generates reflection_limited for Python and copies it to src.
- Added the generated Python for reflection_limited to version control.
- lib/py/setup.py installs reflection_limited.
- lib/py/cleanup.sh now removes build (make sure to run as root).
- The C++ version of reflection_limited was also updated by regen.sh.
Reviewed By: mcslee
Test Plan:
- Clean build and install.
- Look at the installed Python files.
- python -c 'from thrift.reflection.limited.ttypes import *'
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665235 13f79535-47bb-0310-9956-ffa450edef68
Summary:
The Thrift compiler now generates static methods for every service to generate
a reflection of the methods provided by the service. This reflection is fairly
limited, but should be enough for what we want to do with SMC.
Reviewed By: mcslee
Test Plan: test/ReflectionTest.cpp
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665226 13f79535-47bb-0310-9956-ffa450edef68
(This change was by Adam, diff sent by email, I'm just checking it in.)
Blame Rev: 55988
Reviewed By: adam
Test Plan: Thrift compiles.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665219 13f79535-47bb-0310-9956-ffa450edef68
Summary:
using namespace XXX; is tricky because it imports a lot of crap into your namespace.
Also, if we ever decide to remove boost as a dependency, "using namespace boost"
makes it harder.
This patch removes all using namesapce XXX; with one or more using XXX::YYY;
There are some exceptions, such as test, tutorial, namespace std, and
facebook namespaces.
Reviewed By: mcslee
Test Plan:
Recompiled thrift.
grep --binary-files=without-match -r 'using namespace' . | \
egrep -v '^[^:]*: *using namespace (std;$|facebook)' | \
grep -v '<< endl' | \
grep -v '[^:]*/tutorial/' | \
grep -v '^./test/'
Revert Plan: ok
Notes: I wouldn't bother, but it is fixathon.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665218 13f79535-47bb-0310-9956-ffa450edef68
Summary: If using TNonBlockingServer with a ThreadManager, when you send a task off to the threadmanager you need to cancel the event that you have set on that client socket. Otherwise, when you give control back to libevent, it might trigger more read events if there are more requests coming down the pipe. This is an issue, because the server will be in the wrong state at that point and will have no way of handling reading more data if it is still in the WAIT_TASK state trying to see if it should write something back to the client. So, when we hit that control flow, we must setIdle() on the TConnection so that libevent doesn't trigger it anymore. Later, after the result is written, we'll setRead() and go back to the init state.
Reviewed By: akhil
Test Plan: Akhil's async + TNonBlocking karma server
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665217 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Boz is using TMemoryBuffer by constructing it will a NULL buffer,
then doing a resetBuffer later. This patch re-enables that.
Blame Rev: 55988
Reviewed By: aditya
Test Plan:
Thrift compiles.
Will test feed after this is comitted.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665216 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Someone thought it would be a good idea to have two different signatures
for strerror_r, with subtly different semantics (strlcpy = smart).
We now work properly with either of them.
Also fixed a test to work on 32-bit, you sloppy <expletive>s.
Reviewed By: mcslee
Test Plan:
Rebuild thrift.
Force one of these errors to be thrown.
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665215 13f79535-47bb-0310-9956-ffa450edef68
Summary:
TMemoryBuffer already has the necessary plubming to work with C++ strings.
This revision implements that functionality with a few wrapper methods.
Removed TStringBuffer as it should no longer be required (and it is tricky
to safely inherit from a class that has a non-virtual destructor).
Also refactored the TMemoryBuffer constructors a bit.
Reviewed By: aditya, yunfang
Test Plan:
test/TMemoryBufferTest.cpp
Revert Plan: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665213 13f79535-47bb-0310-9956-ffa450edef68
Summary: reinterpret_cast actually breaks on linux, despite working on FreeBSD
Reviewed By: dreiss
Test Plan: Compile on Linux OR FreeBSD
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665211 13f79535-47bb-0310-9956-ffa450edef68
Summary:
/**
* A string buffer is a tranpsort that simply reads from and writes to a
* string. Anytime you call write on it, the data is serialized
* into the underlying buffer, you can call getString() to get the serialized
* string. Before you call read, you should call resetString(data) to set the
* underlying buffer, you can then call read to get the
* de-serialized data structure.
*
* The string buffer is inherited from the memory buffer
* Thus, buffers are allocated using C constructs malloc,realloc, and the size
* doubles as necessary.
*/
Reviewed by: aditya
Test Plan:
int main(int argc, char** argv) {
shared_ptr<TStringBuffer> strBuffer(new TStringBuffer());
shared_ptr<TBinaryProtocol> binaryProtcol(new TBinaryProtocol(strBuffer));
testStruct a;
a.i1 = 10;
a.i2 = 30;
a.s1 = string("holla back a");
a.write(binaryProtcol.get());
string serialized = strBuffer->getString();
shared_ptr<TStringBuffer> strBuffer2(new TStringBuffer());
shared_ptr<TBinaryProtocol> binaryProtcol2(new TBinaryProtocol(strBuffer2));
strBuffer2->resetString(serialized);
testStruct a2;
a2.read(binaryProtcol2.get());
if (a == a2) {
printf("serialization working\n");
} else {
printf("serialization not working\n");
}
}
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665209 13f79535-47bb-0310-9956-ffa450edef68
Summary: add trylock support to the Mutex class
Reviewed By: mcslee
Test Plan: added trylock mutexes to foreman, walked through them in gdb
(behaved correctly) and then ran an entire sweep (behaved correctly)
Notes: slee ftw
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665204 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Replaced a lot of tabs with spaces in the thrift codebase.
Removed a lot of trailing whitespace from thrift-generated c++.
Added a few things to cleanup.sh.
Trac Bug: #
Blame Rev:
Reviewed By: mcslee
Test Plan:
Recompiled thrift.
Re-thrifted some test .thrifts.
Compiled the genrated c++.
Ran cleanup.sh.
Revert Plan: ok
Notes:
EImportant:
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665176 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Added TDebugProtocol, a write-only Thrift protocol for C++
that produces human-readable representations of thrift structs.
Trac Bug: #
Blame Rev:
Reviewed By: mcslee
Test Plan:
Recompiled Thrift.
./test/TestDebugProto.* see compile instructions at the top.
Ran that, and it looked good.
Revert Plan:
grep TDebugProtocol <world>
grep ThriftDebugString <world>
Revert or comment out whatever you find so that they still compile.
Then svn revert.
Notes:
EImportant:
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665166 13f79535-47bb-0310-9956-ffa450edef68
Summary:
scope lock wrapped around ReadWriteMutex. It's like Guard but you can specify to use it as a read or write lock.
Reviewed by: boz
Test Plan: used it in AdFinder
Revertible: yes
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665162 13f79535-47bb-0310-9956-ffa450edef68
Summary: Implemented a read/write lock class in thrift::concurrency
Reviewed By: marc
Test Plan: I compiled this, then compiled a version of fb303 that uses this
(future checkin pending further tests), then ran a binary using the new fb303
and it didn't crash as things were added to the map using the ReadWriteMutex
Revert Plan: yup
Notes: this should replace rw_lock.h for most clients
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665154 13f79535-47bb-0310-9956-ffa450edef68
Summary: TBinaryProtocol.cpp was making assumptions about evaluation order
that are not guaranteed by the standard. Now it does not (at least in these
functions).
Trac Bug: #
Blame Rev:
Reviewed By: mcslee
Test Plan: still compiles.
Revert Plan: svn revert
Notes:
EImportant:
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665152 13f79535-47bb-0310-9956-ffa450edef68
Summary: Wasn't causing any issues because it's a noop with the binary protocol
Reviewed By: jwang
Test Plan: honky tonk, no diff in functionality
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665140 13f79535-47bb-0310-9956-ffa450edef68
Summary: New Thrift TBinaryProtocol with a version identifier
Reviewed By: aditya, eugene
Test Plan: Modify your services to have strictRead_ and strictWrite_ both set to FALSE. Then redeploy your services and test running clients against them. Once you have clients and servers running stably on this new code, you should redploy versions with strictWrite_ set to TRUE. Once that's all good, we can set strictRead_ to TRUE as well, and eventually deprecate the old protocol code entirely.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665138 13f79535-47bb-0310-9956-ffa450edef68
Summary: If enabled, the callback function will be run after the listening socket is opened and the event loop initialized, just prior to calling event_loop()
This is handy if you want to defer some of your initialization until after the socket is open -- allowing clients to connect, but not processing requests until the initialization is complete. I use this in the Synapse tablet server to minimize the communications interruption that happens during a tablet split (replaying commit logs after opening the listening socket).
Review: mcslee
Test Plan: compiled it into Synapse's tablet server
Revert: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665137 13f79535-47bb-0310-9956-ffa450edef68
Summary: If you want requests processed outside of the I/O thread
Reviewed By: jake luciani, aditya
Test Plan: nb-main.cpp, in the test folder
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665132 13f79535-47bb-0310-9956-ffa450edef68
Summary: I'm tired of getting output from thrift via perror AND exceptions, so
this class allows the client to set an alternate (or empty) handler for error
output
Reviewed By: mcslee
Test Plan: I ran on the worker with the default, got output via perror, then
overloaded with my own function and got output via syslog and then NULL
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665131 13f79535-47bb-0310-9956-ffa450edef68
Added explicit detached getter and setter
Modified PosixThreadFactory::~PThread:
Check for join failing and don't transition to detached_ state if it does. Potential thread-handle leak for
threads created joinable who aren't referenced by any external thread. Solution for now has to be
"DONT DO THAT", the clever approach doesn't always work.
Added ThreadFactoryTests.floodNThreads:
Loop M times for N threads where M x N is bigger than 32K to verify that detached threads can be created
ad infinitum.
Reviewed By: mcslee
Revert Plan: revertible
Test Plan: concurrency_test thread-factory passes
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665130 13f79535-47bb-0310-9956-ffa450edef68
Pay attention to detached flags. If thread is create non-detached and has not been joined when all references are given up,
(ie boost::share_ptr calls ~PThread) do the join in the destructor to prevent thread ids from being leaked.
Modified ThreadFactoryTests.reapNThreads:
Loop M times for M threads where M x N is bigger than 32K to verify that thread ids aren't leaked
Modified TimerManager.cpp:
Removed debug messages.
Reviewed By: mcslee
Revert Plan: revertible
Test Plan: concurrency_test thread-factory passes
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665129 13f79535-47bb-0310-9956-ffa450edef68
Summary: Replace with int64_t and don't worry about what architecture machine you're on, the typedefed int64_t will do the right thing.
Reviewed By: aditya, marc
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665123 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- turns out that EAGAIN can be returned both when there is a transmission timeout and when there
is a lack of system resources.
This diff has a hacky fix for respecting a user specified read timeout.
Reviewed By: Steve Grimm, Marc, Slee
Test Plan:
- Tested by trying to crash an srp machine
Revert Plan: No.
Notes:
- Also added functionality to allow users to specify the max number of recv retries (in the case
when EAGAIN is returned due to a lack of system resources)
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665121 13f79535-47bb-0310-9956-ffa450edef68
Throw TimedOutException on wait timeout so caller can distinguish between timeout and event.
Modified facebook::thrift::concurrency::PthreadThread.start:
Throw SystemrResourceException on any pthread_* function call failure rather than asserting 0.
Added facebook::thrift::concurrency::Thread.id() and facebook::thrift::concurrency::ThreadFactory.currentThreadId():
Return thread-id of thread and current thread respectively. Needed for reentrancy tests in ThreadManager
Added facebook::thrift::concurrency::ThreadManager.pendingTaskCountMaxN
Modified facebook::thrift::concurrency::ThreadManager.add():
Now support a maximum pending task count and block if the current pending task count is max.
If timeout is specified for add, TimedOutException is thrown if pending task count doesn't decrease
in the timeout interval. If add() is called by a ThreadManager worker thread and the task cannot
be added, a TooManyPendingTasksException is thrown rather than blocking, since deadlocks can ensue
if worker threads block waiting for works threads to complete tasks.
Reviewed By: mcslee, aditya
Revert Plan: revertible
Test Plan: concurrency/test/ThreadManagerTests.h
run concurrency-test thread-manager
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665120 13f79535-47bb-0310-9956-ffa450edef68
Summary: SO_REUSEADDR doesn't always work, and some of my servers have
expensive init routines that I really want to run before it becomes available
to services, so I want thrift to retry the bind several times at varied
intervals before giving up
Reviewed By: slee
Test Plan: running a service in GDB with no problems binding, with another
service bound to the port for all the retries, another service bound to the
port for only some of the retries
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665109 13f79535-47bb-0310-9956-ffa450edef68
