Image processing server using NodeJS

I have a NodeJS server serving web requests, users can upload images to my server as well. I am currently processing these images on the same server. It was really a decision to end whitespace until I could do something better as image processing is CPU intensive and I don't want to do this on my web server.

The type of processing is very simple:

  • Auto-orient the image, send the image to the processing server and return the oriented image to it.
  • Create thumbnails, send the image to the processing server and execute it returns a thumbnail.

I would like to be able to use this imaging server for many other servers. I also want it to work with small memory / cpu. I.E. if I have more CPU / memory available, but I don't want it to crash if deployed on a server with limited resources.

The way I'm currently handling this, I have a config file that defines how many images I can handle at the same time. If I have very limited resources, I will only process one image at a time while staying in the queue. This works great now because the web server and the imaging server are the same.

My standalone imaging server will have a "queue" of requests so that it can limit the number of images processed at a time.

I don't want to store images on the processing server. Mainly because my web server knows the data structure (which images belong to which content). So I really want to just make a request to the server and return the processed image.

Some questions.

  • Is the NodeJS implementation a good solution?
  • If it is going to be RESTful http api. I'm curious about this because I'm not sure if I can do this given the requirement to only process a certain number of images at a time. If I get 15 requests at the same time, I'm not sure what to do here, I don't want to process the first one while others are waiting for a response (processed image).
  • It is possible to use sockets to connect my webserver to my imaging server to avoid question # 2. This is not as "revealing" as it can be more difficult to use with other servers. But pushes me out of the request / response problem I have in # 2 above. The IE web server can invoke image processing on the socket, and the processing server can simply respond to the socket when it is done.
  • Is there some open source solution that will work on linux / unix that will suit my needs. This seems like a fairly common problem, it should be fun to implement, but wouldn't want to reinvent the wheel if I could use / contribute to another product. I have seen Apace :: ImageMagick but I cannot get the POST images they will have to host on the server already. There are also some windows that I cannot use.

Please, help...

+3


source to share


1 answer


Why not break the process down into discrete steps.

  • User uploads an image. Receives some kind of confirmation message that the image has been successfully loaded and processed.

  • Place an image processing request in the queue.

  • Perform image processing.

  • Notify the user that image processing is complete.



This suits your requirements in that users can upload many images in a short period of time without overloading the system (they just enter the queue), image processing can take place on a separate server without moving files)

0


source







All Articles