How to make sure we read all lines from the boost :: child process
I saw the following code on the documentation page boost::child
where they explain how to read the output of the child process.
http://www.boost.org/doc/libs/1_64_0/doc/html/boost_process/tutorial.html
They say that after starting your child process, we can read it through this loop: -
bp::ipstream is; //reading pipe-stream
bp::child c(bp::search_patk("nm"), file, bp::std_out > is);
//then later
while (c.running() && std::getline(is, line) && !line.empty())
data.push_back(line);
I have 2 questions: -
- If it
c.running()
returns false, we just exit the loop. And in this case, the streamis
above could still carry data that is lost? - What is the best way to read stdout and stderr, while making sure the exit () process does not create a deadlock There is a warning on the page: -
Pipe will deadlock if you try to read after exiting
I want to record both stdout
, and stderr
without worrying about what nm
came out or was not higher.
source to share
I think there is no right way unless you are using async methods.
Maybe you can just get the future for the vector and use string_views in that if you somehow really need it in turn.
std::future<std::vector<char> > output, error;
boost::asio::io_service svc;
bp::child c(bp::search_path("nm"), file, bp::std_out > output, bp::std_err > error, svc);
svc.run();
To read exactly as you did before you can use istream on top of a vector:
#include <boost/process.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream_buffer.hpp>
#include <iostream>
namespace bp = boost::process;
namespace bio = boost::iostreams;
std::string const file = "./a.out";
int main() {
std::future<std::vector<char> > output, error;
boost::asio::io_service svc;
bp::child c(bp::search_path("nm"), file, bp::std_out > output, bp::std_err > error, svc);
svc.run();
//then later
{
auto raw = output.get();
std::vector<std::string> data;
std::string line;
bio::stream_buffer<bio::array_source> sb(raw.data(), raw.size());
std::istream is(&sb);
while (std::getline(is, line) && !line.empty())
data.push_back(line);
std::cout << data.at(rand()%data.size()) << "\n";
}
}
source to share
I had the same problem ... The best way to deal with this is using async i / o.
Unfortunately the boost @ documentation http://www.boost.org/doc/libs/master/doc/html/boost_process/extend.html#boost_process.extend.async was wrong ... It makes everything look simple but not shows that buffers should be predefined and masked over a lot of details.
Here's my function that sends or receives one buffer at a time (no interaction like question / response0, I use stderr to check for errors as it was necessary for my application, but you could catch the application exit code by calling "c. exit_code () ;. `
using tstring=basic_string<TCHAR>;
void Run(
const tstring& exeName;
const tstring& args,
const std::string& input,
std::string& output,
std::string& error
)
{
using namespace boost;
asio::io_service ios;
std::vector<char> vOut(128 << 10);
auto outBuffer{ asio::buffer(vOut) };
process::async_pipe pipeOut(ios);
std::function<void(const system::error_code & ec, std::size_t n)> onStdOut;
onStdOut = [&](const system::error_code & ec, size_t n)
{
output.reserve(output.size() + n);
output.insert(output.end(), vOut.begin(), vOut.begin() + n);
if (!ec)
{
asio::async_read(pipeOut, outBuffer, onStdOut);
}
};
std::vector<char> vErr(128 << 10);
auto errBuffer{ asio::buffer(vErr) };
process::async_pipe pipeErr(ios);
std::function<void(const system::error_code & ec, std::size_t n)> onStdErr;
onStdErr = [&](const system::error_code & ec, size_t n)
{
error.reserve(error.size() + n);
error.insert(error.end(), vErr.begin(), vErr.begin() + n);
if (!ec)
{
asio::async_read(pipeErr, errBuffer, onStdErr);
}
};
auto inBuffer{ asio::buffer(input) };
process::async_pipe pipeIn(ios);
process::child c(
exeName + _T(" ") + args,
process::std_out > pipeOut,
process::std_err > pipeErr,
process::std_in < pipeIn
);
asio::async_write(pipeIn, inBuffer,
[&](const system::error_code & ec, std::size_t n)
{
pipeIn.async_close();
});
asio::async_read(pipeOut, outBuffer, onStdOut);
asio::async_read(pipeErr, errBuffer, onStdErr);
ios.run();
c.wait();
}
The application I am running is a decoder / encoder, I send the whole file to be processed and get the results at the same time. File size limitation.
IMPORTANT
Fixed a bug in boost 1.64, it only affects Windows, apparently
in boost \ process \ detail \ windows \ async_pipe.hpp: ref: https://github.com/klemens-morgenstern/boost-process/issues/90
line 79:
~async_pipe()
{
//fix
//if (_sink .native() != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
// ::boost::detail::winapi::CloseHandle(_sink.native());
//if (_source.native() != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
// ::boost::detail::winapi::CloseHandle(_source.native());
boost::system::error_code ec;
close(ec);
//fix
}
source to share