Does boost::asio truncate output when using async_read_until?_问答_开发者

This one has me stumped!

There seems to be a problem printing out anything over 4k bytes if I use the async_read_until call?

I have a little function that prints out 100 lines (just over 4k).

Works fine in every combination except if I register a async_rea开发者_如何转开发d_until callback. At that point my output is truncates to about 4k. Note not always though, sometimes less and sometimes the whole thing is printed it seems to be related to the load on the machine, almost like there is some timeout going on? Some asio threading thing? Anyway if I comment out the async_read_until call it works fine every time no matter how many time I call printLines. I can even use the ioService post function it works fine...

What is going on? BTW, I'm using linux and amd64 machine gcc4.4. (Redhat)

Using linux 'strace' I got some more clues:

It seems like after calling async_read_until, asio using fcntl calls, causes my output file descriptor to change behaviour?

it quits printing output after awhile:

select(4, [0 3], [], [], {300, 0}) = 1 (in [0], left {298, 830000})

readv(0, [{"\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512}], 1) = 1

write(2, "This is a test of a long sentenc"..., 93) = 93

... about 40 times

write(2, "This is a test of a long sentenc"..., 93) = 53

write(2, "This is a test of a long sentenc"..., 93) = -1 EAGAIN (Resource temporarily unavailable)

... for the rest until 100 is reached.

So you can see the select loop waiting for my input return key. Then we call my empty read handler and drop out of the ioService. At this time I call my printLines function and try to print 100 lines but it quits after printing 40 some.

Something with this EAGAIN is causing the output to stop writting.

Again if I don't call async_read_until my printf does not get corrupted.

I think I know what is going on, seems like Asio is turning my stdout file desciptor into async non-blocking mode when I request async read mode on the stdin file descriptor. That's why after some output I get EAGAIN errors on writes. Of course printf ignores those so my output is truncated. Don't know if that is a bug in Asio or just a side effect on linux?

Here is my simple program to duplicate the problem:

#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <iostream>

using namespace boost::asio;
using namespace std;

io_service ioService;
boost::asio::streambuf inStream;
posix::stream_descriptor input(ioService, STDIN_FILENO);

void printLines()
{
    for (int i = 0; i < 100; i++) {
        fprintf(stderr, "This is a test of a long sentence, there will be %d more sentences after this on is printed.\n", i);
    }
    fflush(stderr);
}

void readHandler(const boost::system::error_code& error)
{ // Don't care about read!! }

int main()
{
    boost::system::error_code ec;
    //printLines(); << this works if uncommented
    //ioService.post(printLines); << this works if uncommented

    boost::asio::async_read_until(input, inStream, "\n",
    bind(readHandler, placeholders::error)); // causes truncated output

    cout << "Hit Return to continue..." << endl;
    ioService.run_one( ec );
    assert(!ec);
    printLines(); // partial output if async_read_until is called?
    return 0;
}

If I insert the following call to ioctl prior to printLines

int opt = 0;
ioctl( STDIN_FILENO, FIONBIO, &opt );
printLines();

the behavior is expected

samm@macmini ~> ./a.out
Hit Return to continue...

This is a test of a long sentence, there will be 0 more sentences after this on is printed.
...
This is a test of a long sentence, there will be 96 more sentences after this on is printed.
This is a test of a long sentence, there will be 97 more sentences after this on is printed.
This is a test of a long sentence, there will be 98 more sentences after this on is printed.
This is a test of a long sentence, there will be 99 more sentences after this on is printed.
samm@macmini ~>

Though it's not clear to me why this happens, it could be a side effect of ioctl, which is largely kernel/device specific. I did not see anything describing this behavior when trolling around the various man pages. You might try switching from using fprintf in printLines to using a posix::stream_descriptor with STDERR_FILENO instead.

Edit: it looks like there have been some recent changes to Boost.Asio that may address this behavior. Specifically

Added new non_blocking() functions for managing the non-blocking behaviour of a socket or descriptor. The io_control() commands named non_blocking_io are now deprecated in favour of these new functions.

Added new native_non_blocking() functions for managing the non-blocking mode of the underlying socket or descriptor. These functions are intended to allow the encapsulation of arbitrary non-blocking system calls as asynchronous operations, in a way that is transparent to the user of the socket object. The functions have no effect on the behaviour of the synchronous operations of the socket or descriptor.

look interesting to me. I used Boost 1.45 on my mac to run your reproducer, and these changes will be in the upcoming Boost 1.47. I'll try to grab Chris's latest asio development release and see if the behavior has changed.