Streaming Data Using Netflix Feign

At work, we use the Netflix Feign Client to help with cross-service requests. However, I'm confused by my apparent inability to transfer data, especially given Netflix's famous business model for streaming video. I am clearly missing something.

To explain, let's say Service A

a Feign client is requesting Service B

for a data stream and Service B

sending a stream in response. At this point, the method execute()

in the front client is called:

@Override public Response execute(Request request, Options options) throws IOException {
  HttpURLConnection connection = convertAndSend(request, options);
  return convertResponse(connection);
}

HttpURLConnection convertAndSend(Request request, Options options) throws IOException {
  final HttpURLConnection connection = (HttpURLConnection) new URL(request.url()).openConnection();

  /** SNIP **/

  if (request.body() != null) {
    if (contentLength != null) {
      connection.setFixedLengthStreamingMode(contentLength);
    } else {
      connection.setChunkedStreamingMode(8196);
    }
    connection.setDoOutput(true);
    OutputStream out = connection.getOutputStream();
    if (gzipEncodedRequest) {
      out = new GZIPOutputStream(out);
    }
    try {
      out.write(request.body()); // PROBLEM
    } finally {
      try {
        out.close();
      } catch (IOException suppressed) {
      }
    }
  }
  return connection;
}

      

The line marked PROBLEM

confuses me.

  • The object request

    doesn't even have any kind of stream to read, just byte[] body

    .
  • At the original end, the entire body is written in at OutputStream

    once. Shouldn't this be the data instead?

for example

// pseudocode
try {
  location = 0
  bufferSize = 2048
  buffer = request.body().read(location, bufferSize)
  while(out.readyToRead() && buffer.length > 0) {
    out.write(buffer)
    location += bufferSize
    buffer = request.body().read(location, bufferSize)
  }
}

      

If there was a thread in the request, not just one byte[] body

, you could improve that even further to send data as it comes in.

I am very new to this area of ​​service architecture. What am I missing?

+3


source to share


1 answer


Feign was designed for a control plane apis, which is often not beneficial if flows up. Downstream streaming is however supported.

I am not worried about being more efficient about how buffering works (like a byte array alternative). Just keep in mind that most of the simulator design revolves around template templates (json or xml) and uses them as much as possible (e.g. on retransmission, buffering + fixed length easy and predictable).



I think I would be very happy with the "streaming" design if it was bundled with an http client. IOTW, a subtype that refers to a stream in a way that makes sense in transport. For example, InputStream for regular java, OkIo buffer for OkHttp, Netty Buffer for Netty, etc.

Spencer opened this up for investigation https://github.com/Netflix/feign/issues/220

+5


source







All Articles