Summary: Wasn't causing any issues because it's a noop with the binary protocol
Reviewed By: jwang
Test Plan: honky tonk, no diff in functionality
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665140 13f79535-47bb-0310-9956-ffa450edef68
Summary: New Thrift TBinaryProtocol with a version identifier
Reviewed By: aditya, eugene
Test Plan: Modify your services to have strictRead_ and strictWrite_ both set to FALSE. Then redeploy your services and test running clients against them. Once you have clients and servers running stably on this new code, you should redploy versions with strictWrite_ set to TRUE. Once that's all good, we can set strictRead_ to TRUE as well, and eventually deprecate the old protocol code entirely.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665138 13f79535-47bb-0310-9956-ffa450edef68
Summary: If enabled, the callback function will be run after the listening socket is opened and the event loop initialized, just prior to calling event_loop()
This is handy if you want to defer some of your initialization until after the socket is open -- allowing clients to connect, but not processing requests until the initialization is complete. I use this in the Synapse tablet server to minimize the communications interruption that happens during a tablet split (replaying commit logs after opening the listening socket).
Review: mcslee
Test Plan: compiled it into Synapse's tablet server
Revert: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665137 13f79535-47bb-0310-9956-ffa450edef68
Summary: If you want requests processed outside of the I/O thread
Reviewed By: jake luciani, aditya
Test Plan: nb-main.cpp, in the test folder
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665132 13f79535-47bb-0310-9956-ffa450edef68
Summary: I'm tired of getting output from thrift via perror AND exceptions, so
this class allows the client to set an alternate (or empty) handler for error
output
Reviewed By: mcslee
Test Plan: I ran on the worker with the default, got output via perror, then
overloaded with my own function and got output via syslog and then NULL
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665131 13f79535-47bb-0310-9956-ffa450edef68
Added explicit detached getter and setter
Modified PosixThreadFactory::~PThread:
Check for join failing and don't transition to detached_ state if it does. Potential thread-handle leak for
threads created joinable who aren't referenced by any external thread. Solution for now has to be
"DONT DO THAT", the clever approach doesn't always work.
Added ThreadFactoryTests.floodNThreads:
Loop M times for N threads where M x N is bigger than 32K to verify that detached threads can be created
ad infinitum.
Reviewed By: mcslee
Revert Plan: revertible
Test Plan: concurrency_test thread-factory passes
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665130 13f79535-47bb-0310-9956-ffa450edef68
Pay attention to detached flags. If thread is create non-detached and has not been joined when all references are given up,
(ie boost::share_ptr calls ~PThread) do the join in the destructor to prevent thread ids from being leaked.
Modified ThreadFactoryTests.reapNThreads:
Loop M times for M threads where M x N is bigger than 32K to verify that thread ids aren't leaked
Modified TimerManager.cpp:
Removed debug messages.
Reviewed By: mcslee
Revert Plan: revertible
Test Plan: concurrency_test thread-factory passes
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665129 13f79535-47bb-0310-9956-ffa450edef68
Summary: Replace with int64_t and don't worry about what architecture machine you're on, the typedefed int64_t will do the right thing.
Reviewed By: aditya, marc
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665123 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- turns out that EAGAIN can be returned both when there is a transmission timeout and when there
is a lack of system resources.
This diff has a hacky fix for respecting a user specified read timeout.
Reviewed By: Steve Grimm, Marc, Slee
Test Plan:
- Tested by trying to crash an srp machine
Revert Plan: No.
Notes:
- Also added functionality to allow users to specify the max number of recv retries (in the case
when EAGAIN is returned due to a lack of system resources)
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665121 13f79535-47bb-0310-9956-ffa450edef68
Throw TimedOutException on wait timeout so caller can distinguish between timeout and event.
Modified facebook::thrift::concurrency::PthreadThread.start:
Throw SystemrResourceException on any pthread_* function call failure rather than asserting 0.
Added facebook::thrift::concurrency::Thread.id() and facebook::thrift::concurrency::ThreadFactory.currentThreadId():
Return thread-id of thread and current thread respectively. Needed for reentrancy tests in ThreadManager
Added facebook::thrift::concurrency::ThreadManager.pendingTaskCountMaxN
Modified facebook::thrift::concurrency::ThreadManager.add():
Now support a maximum pending task count and block if the current pending task count is max.
If timeout is specified for add, TimedOutException is thrown if pending task count doesn't decrease
in the timeout interval. If add() is called by a ThreadManager worker thread and the task cannot
be added, a TooManyPendingTasksException is thrown rather than blocking, since deadlocks can ensue
if worker threads block waiting for works threads to complete tasks.
Reviewed By: mcslee, aditya
Revert Plan: revertible
Test Plan: concurrency/test/ThreadManagerTests.h
run concurrency-test thread-manager
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665120 13f79535-47bb-0310-9956-ffa450edef68
Summary: SO_REUSEADDR doesn't always work, and some of my servers have
expensive init routines that I really want to run before it becomes available
to services, so I want thrift to retry the bind several times at varied
intervals before giving up
Reviewed By: slee
Test Plan: running a service in GDB with no problems binding, with another
service bound to the port for all the retries, another service bound to the
port for only some of the retries
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665109 13f79535-47bb-0310-9956-ffa450edef68
Summary: Identical to string in all languages except Java. Java String is NOT binary-safe, so we need to use raw byte[] in that case. PHP/RUBY/Python strings are all binary safe, and C++ std::string works fine and manages memory for you so it's the safest route. Java just needs this tweak.
Reviewed By: aditya
Test Plan: Use "binary" as a type instead of String.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665099 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- Make sure that buffers are always emptied when destructor is called (or when file is closed)
- make flush a no-op if file is not open
- dont allow new events to be enqueued if the file is in the process of being closed
Reviewed By: slee, jwang
Test Plan: tested with building search indices on top of thrift logs
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665094 13f79535-47bb-0310-9956-ffa450edef68
Summary: Not including config.h means we never get the HAVE_CLOCK_GETTIME #define, breaking the build with a duplicate function definition on machines that have a clock_gettime() function in the standard library.
Reviewed By: slee
Test Plan: it compiles, ship it
Revert: ok
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665087 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- cannot assume that assert.h is defined
Reviewed By: marc k
Test Plan: everything compiles
Notes:
- need to reflect these changes in libfacebook/fbthread
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665073 13f79535-47bb-0310-9956-ffa450edef68
- StatsProcessor can be used to print events, or keep track of event frequency
- PeekProcessor is used to examine data in a thrift event, prior to passing it along to an underlying processor
- TPipedFileReaderTransport and its factory are used to pipe a TFileReaderTransport (which TFileProcessor requires)
Also fixed some bugs in TFileTransport - next flush time was overflowing and not always being reset
Reviewed by: aditya, mcslee
Test Plan: Tested using various thrift clients (scribe, falcon) and gdb in sandbox and on dev008.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665066 13f79535-47bb-0310-9956-ffa450edef68
Summary: Now you can join against all the threads in a ThreadManager
Reviewed By: marc, xp-style
Test Plan: Use with new ThriftServer shutdown mechanisms
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665037 13f79535-47bb-0310-9956-ffa450edef68
Summary: When reading chunked encoding don't always get all the parts right away, return one part at a time so that reading code can deserialize the data on the fly.
Reviewed By: http
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665025 13f79535-47bb-0310-9956-ffa450edef68
Reviewed By: aditya
line, and those below, will be ignored--
M php/src/Thrift.php
M php/src/transport/TSocket.php
M php/src/transport/TPhpStream.php
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665021 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- I love finding overflow errors.
Reviewed By: jwang
Test Plan: tested by rolling back search logfile
Notes:
- This should not affect anything currently in production.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665020 13f79535-47bb-0310-9956-ffa450edef68
Summary: Add application exceptions for unknown methods etc, and also let binary protocol support size limits on containers and strings
Reviewed By: aditya, xp-wayne
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665003 13f79535-47bb-0310-9956-ffa450edef68
Summary: Too irritating not having stack traces when these go down. Sack up and include stack traces always... bigger binary but whatever.
Reviewed By: aditya
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@665002 13f79535-47bb-0310-9956-ffa450edef68
Reviewed by: aditya
Tested with thrift test class
Notes: TFileTransport used to use a circular buffer. Changed this to use two large buffers, one for reading and one for writing, that are swapped whenever the writer thread finishes with the last write.
Also changed a few default constants -- force_flush timeout is now 3 sec, default buffer size is 10000 entries
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664997 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Sockets now error with timeout messages if it's a timeout over general read failures.
Reviewed By: Slee
Test Plan: Tested with payment broker client with overloaded addresses and send/recv timeouts.
Revert Plan: revertible
Notes:
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664993 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Thrift improperly initialized file descriptors to zero and treated 0 and an invalid value. 0 is a valid file descriptor. Also, commented out error message on client disconnect for TThreadPoolServer
Reviewed By: Marc & Karl
Revert Plan: revertible
Notes:
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664972 13f79535-47bb-0310-9956-ffa450edef68
Summary:
TServerSocket detected file des. 0 as an invalid file des. This has been fixed.
Reviewed By: Steve
Revert Plan: Revertible
Notes:
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664969 13f79535-47bb-0310-9956-ffa450edef68
Summary: libthriftnb is the one that requires libevent. It was really annoying that you always had to put -levent in your link step to build thrift programs when youa re not using any of the c++ nonblocking stuff. This way, if you're using nonblocking you can do that, but the normal case is easier to link.
Reviewed By: makefile
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664951 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- perform some basic corruption checks:
1) Event larger than chunk
2) Event larger than specified max
3) Event crossing chunk boundary etc.
- If error encountered, then try to perform some recovery
Reviewed By: Slee
Test Plan: Going to test now...need to check in because of compile issues
Notes:
- These checks take care of the case when there is a dirty read from the filesystem (which
we have encountered with the netapps). The recovery involves trying to perform the read
again from ths FS and if that fails skipping the chunk altogether.
Keep in mind that this might only be useful for idempotent systems (e.g. search redolog).
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664943 13f79535-47bb-0310-9956-ffa450edef68
Summary:
- Transport factories now wrap around one transport
- Protocol factories now wrap around one transport (as opposed to a pair of input/output
transports)
- TServer now takes input/output transport and protocol factories
The motivation for this change is that you could concievably want to use a different protocol or
transport for input and output. An example is that incoming data is encoded using binary protocol
but outgoing data is encrypted XML (with encryption being done on the transport level).
This change should be mostly backwards compatible because the TServer classes have constructors
that take a transport factory and use that for both the input and transport factories. The only
change might be for anyone who is using the C++ client code directly i.e. instantiating
TBinaryProtocol() directly because the constructor now only accepts one transport.
Reviewed By: Slee
Test Plan: Everything compiles (for both thrift and search).
Notes:
I am going to make the same changes in all the supported languages after this...
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664940 13f79535-47bb-0310-9956-ffa450edef68
Summary:
-- the constructor needs to accept a transport factory
-- TConnection close() needs to close factor generated transports
Reviewed By: Mark Slee
Test Plan: Tested with search redologger
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664930 13f79535-47bb-0310-9956-ffa450edef68
Trac Bug: #
Reviewed By:
Test Plan:
The correct error message appears when a PHP Thrift client can't
connect to a server. Set that up, and try and get the error message:
[Wed Jan 17 10:41:41 2007] [error] TSocketPool: retryInterval (60) has
passed for host dev007:8188
...then you're good. If you get something about a undefined property,
then this fix didn't work.
Revert Plan: Revertible
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664920 13f79535-47bb-0310-9956-ffa450edef68
Summary:
-- TBufferedFileWriter.h/cpp will be renamed to TFileTransport.h/cpp in the next commit.
-- TFileTransport is essentially reading and writing thrift calls to/from a file instead of a
socket.
-- The code/design is somewhat similar to pillar_logfile but there are some significant changes.
todo:
-- still need to do error correction/detection
Reviewed By: Mark Slee
Test Plan:
-- Wrote test in thrift/test/cpp/src/main.cpp that appends to a file and replays requests
Notes:
It's finally time to port search over to Thrift
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664889 13f79535-47bb-0310-9956-ffa450edef68
Summary:
Clients would elect to call the base class what() method instead of the derived version. This results in
less than usable exceptions for debugging.
Reviewed By: Karl * mcslee
Test Plan:
Tested with local application in development.
Revert Plan:
revertible.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664887 13f79535-47bb-0310-9956-ffa450edef68
Summary: If you read the string "0" then it treats that as false, so we have to check for both === false and === '' manually.
Reviewed By: martin
Notes: I hate php
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664878 13f79535-47bb-0310-9956-ffa450edef68
Summary:
-- This is the thrifty version of Pillar's batch_writer
-- Cleaned up a lot of the code in batch writer and made it conform to Thrift's strict coding standards
-- Added TBufferedRouterTransport.h/cc to actually route messsages via readEnd() to the file writer. It's
not quite as easy to route the messages in Thrift as it was in Pillar
Reviewed By: Slee
Test Plan: Tested by making sure that the file was recording data
Notes:
-- The real correctness test will be when I finish writing TLogFileTransport (pillar_logfile.cpp).
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664826 13f79535-47bb-0310-9956-ffa450edef68
Summary:
-- Made read buffering work (the buffer wasn't actually being used)
-- TServer now extends TRunnable (useful for testing)
-- Adding TLogging (the DEBUG and ERROR macros that we all know and love)
Reviewed By: Slee
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664824 13f79535-47bb-0310-9956-ffa450edef68
Summary: Servers need to create bufferedtransports etc. around the transports they get in a user-definable way. So use a factory pattern to allow the user to supply an object to the server that defines this behavior.
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664792 13f79535-47bb-0310-9956-ffa450edef68
Summary: Just for completeness cause I'm crazy. Let's never use these!
Notes: Also made thrift grammar support # style comments, so you can do this at the top of your files
#!/usr/local/bin/thrift --cpp
/**
* This is a thrift def file youc an invoke directly and gen code!
*/
blah
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664789 13f79535-47bb-0310-9956-ffa450edef68
Summary:
-- these are hook points that thrift_logger/logfile will integrate with..
Reviewed By: McSlee
Test Plan: Generated thrift code
Notes:
-- Thrift logfiles are soon coming to a netapp near you!
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664787 13f79535-47bb-0310-9956-ffa450edef68
Summary: Yep, it's up and running. We now have full client/server support in all of C++ Java PHP and Python. Well, not quite... there's no PHP server, but honestly who wants one? Actually, if we do want one the framework will support writing is as a PHP file that can be served in apache like a web service (i.e. restserver.php would be thriftserver.php). But now that's rambling and nothing to do with this commit.
Notes: cheever, let's chat about porting your multithreaded Pillar Python server over to Thrift
git-svn-id: https://svn.apache.org/repos/asf/incubator/thrift/trunk@664783 13f79535-47bb-0310-9956-ffa450edef68
