Choosing the right AWS services and software
I am developing an IoT application prototype that does the following
- Receive / save data from sensors.
- A web application with a web IDE for users to deploy simple JavaScript / Python scripts that run in Docker containers.
- The data from the sensors are transferred to these containers.
- User programs can use this data for analysis, monitoring, etc.
- The logs of these programs are displayed to the user in the webapp
Current architecture and services
- Using one instance of AWS EC2. I chose EC2 because I was trying to figure out the architecture.
- Node.js stack, RabbitMQ, Express, MySQl, MongoDB and Docker
- I don't need AWS IoT services like AWS IoT and Greengrass.
- I have excluded Heroku as I am using other AWS services.
Questions and problems
My goal is to prototype a beta for 50 users . (hope someone else helps / will work on releasing the product)
- As much as possible, I don't want to spend a lot of time migrating between services as product development is key. Should I stick with EC2 or switch to Beanstalk?
- If I am sticking to EC2, what is the best way to handle small to medium traffic? Use one large EC2 computer or several small microcontrollers?
- What's a good way to manage containers? Is it worth using a swarm and managing containers? What if I need to use multiple instances?
- I also have small scripts that have a status of sensor information that the web app and other services need. If I go to multiple instances, how can I make these scripts available to multiple machines?
- This question also applies to servers, message buses, databases, etc.
My goal is not production. I want to complete the product, show that I have users who are interested and of course show that the product works!
Any help in this regard would be really appreciated!
source to share
If you want to manage your docker containers with the least hassle on AWS, you can use Amazon ECS to deploy your containers or using Beanstalk. Also you don't need to use Swarm on AWS, ECS will work for you.
It's always best to scale, and scale using small to medium EC2 instances. The challenge you will face, however, is managing and scaling the underlying EC2 as well as your docker containers. This leads to the use of large EC2 instances to keep EC2 scaling aside and focus on docker scaling (which will add extra cost to you).
Another alternative that you can use for the web application part is to use the AWS Lambda stack and API Gateway with Serverless Framework , which should have the least operational overhead and comes with DevOps tools.
source to share
You can save your web app to Heroku and run your IoT server on AWS EC2 or AWS Lambda. Heroku is on AWS, so this split setup won't have a performance impact. You can fix this "sitting on two chairs" inconvenience by writing a Terraform script that contains both the EC2 instance and the Heroku application and links them together.
Alternatively, you can use the Dockhero add-on to run your IoT server in a Docker container along with your Heroku app.
ps: I support Dockhero
source to share