Best setup to work with amazon AWS
I have a website that gets a backup from various social networks and then saves the data to a server and then displays on my website. content includes video, images and text data.
I am currently using an EC2 instance with RDS and EBS. The data is stored in EBS volumes, but since the amount of data is large enough, more than 1 TB, and this is increasing. Every time my EBS volume fills up, I add another volume.
Then I added S3 to my installation. Cron jobs run and store data in S3, and an EC2 instance displays data from S3. I am using PHP SDK for this.
The problem I'm running into is that S3 is very slow in my current setup. Please suggest if my setup is good or if I need some change in my setup and how can I speed up S3. or should I choose another method for my installation. The EC2 Instance is a large Reserved Instance that runs CentOS.
I've listened to some of the S3fs that mount the S3 bucket in Ec2 as a volume. This is a good choice as when I installed the S3 Bucket instance in Ec2 the transfer rates were very slow.
I'm new to AWS. My users don't access files directly from S3, but they access my website, which is running on an EC2 instance.
source to share
RDS is a good choice for storing metadata such as tags, comments, and other relevant information about your media files. The S3 is good for storing static content like Video, Audio, and Pictures. I think your approach with RDS and S3 is good enough.
Supported EBS instances are good for persistence. If you are storing your metadata in RDS and static content on S3, the only reason you should be using EBS supported by EC2 instances is because you have some configuration files that are not checked back now. If this is not the case, if you think your configuration is versioned and is available on demand for a new instance each time, you may want to strip out the EBS volumes in favor of ephemeral storage. It might give you some performance boost, but nothing significant.
As for your S3 latency, yes, S3 is slow. While all of your recordings can originate directly from S3, I highly recommend that you configure Amazon CloudFront for your S3 buckets and let your website consume media content from CloudFront. CloudFront is a content delivery network (CDN) that works with disk volumes (either EBS-supported or ephemeral) as well as S3. This will take no more than a few minutes. CloudFront also supports streaming media over RTMP . You may need a library like GPACto prompt multimedia files to make them more flexible if they are not already running. Then you might want to create one distro for video / audio for streaming and another distro for images, Javascript, styles, and other text files.
Hope it helps.
source to share
To get and download files from Amazon S3 faster, I use the one batch()
found here .
You can also use cloudfront to get files faster. I think 9gag uses cloudfront as well.
source to share