We are the Amazon EC2 Container Service team – AUAA!

There is a bug currently in the console, if you try to "Add Container definition" with a small screen (13 inch laptop), the window to add the container is too low, and we cannot access the buttons to complete the form. I have send feedback but haven't heard back.

What's the best way to feed that back and get it fixed?

Thank you. We are aware of this bug and are in the process of fixing it and will be rolling it out soon.

Q1. Any plans to integrate ECS with Lambda? I.e. ability to map Lambda event sources to ECS task/service instead of Lambda function?

Q2. Any plans to implement cron-like scheduler for ECS tasks, either native or via external schedulers like AirBnB Chronos?

Q1. That does sound super cool. We'll take that back to the Lambda team.

Q2. We have a Mesos Scheduler Driver than allows you to use Chronos with ECS (https://github.com/awslabs/ecs-mesos-scheduler-driver). Note that this is just a proof of concept to show how you can do this. We've heard this request from others customers as well and are looking at how we can make this even easier.

Welcome to IAMA. We're happily using EC2 for a couple of our behind the scenes bots here - so thanks!

Other than the IAMA bots, what's the coolest thing going on in your datacenters? What kinds of crazy technology are you using to run what feels like half of the modern internet?

Hi, Thank you for your question. There are a number of cool things going on in our datacenters. Instead of listing all of them, let me point you to a couple of talks from the last re:Invent. The first one is James Hamilton's talk on Innovation at Scale: https://www.youtube.com/watch?v=JIQETrFC_SQ and the other one is a talk on some of the core distributed systems that we use at AWS: https://www.youtube.com/watch?v=QVvFVwyElLY

Amazon EC2 container service team, what's your favourite cheese?

Lots of folks here seem to like Harvati. Brie and Gouda are right on the heels. Jeff Barr seems to like smelly ones and Sam likes Beecher's Flagship.

What kind of monitoring is available today for ECS and can you tell us what will be coming soon?

Also interested in seeing a dashboard which tells me where my resources are and what they're up to. At scale, any of the docker management solutions will need to make it simple to track patterns.

Thanks for your feedback. You can send custom metrics to cloudwatch, e.g. from cAdvisor today. We've also heard from customers that they are including a log pulling and metrics generation container in their Task Definitions

We are also interested in the kinds of metrics you would be interested in and how you would use them. If you could post these on the ECS forums that would be super awesome (https://forums.aws.amazon.com/forum.jspa?forumID=187)

Is there a way to get the cluster to scale on demand, à la beanstalk?

If not, any plan for the the near future?

Multi-container Docker environments for Beanstalk leverage ECS. So you can continue to use that. But if you want to use ECS, you can use AutoScaling to scale your fleet of EC2 instances. Keep an eye out on our blog for examples of how you can do that.

Multi-container Docker environments for Beanstalk

Wow! How did I miss this:

http://aws.amazon.com/about-aws/whats-new/2015/03/aws-elastic-beanstalk-supports-multi-container-docker-environments/

If you're going to use ECS anyway, are there any advantage to Docker in Beanstalk rather than ECS?

Elastic Beanstalk is what you want to use of you want the simplicity of deploying applications within the Beanstalk application model. If you are running multiple applications across a fleet of instances or want more control over the application architectre, ECS will give you more flexibility.

What's your blog URI?

http://aws.amazon.com/blogs/compute

http://aws.amazon.com/blogs/aws

Hi. Thank you for the great service, guys,

First of all, I'd like to tell that I had to wake up at 4 am in order to ask a bunch of questions. We like ECS very and are thinking of moving our architecture to Microsevices.

Question: Auto Scaling Group

It can spin up and shut down EC2 instances. Will ASG be able to spin up and shut down containers (TaskDefinitions) dynamicaly? When?

Related StackOverflow question I asked: http://stackoverflow.com/questions/29737034/does-aws-ecs-support-per-container-dynamic-scalability

Thank you for getting up in the wee hours. Certainly appreciate that.

ECS has a CreateService and UpdateService API for long running tasks. For tasks started with CreateService you can trigger UpdateService based on the metrics you care about. Keep an eye out on our blog for examples of how you can scale tasks and integrate with Auto Scaling.

I quickly browsed the docs for CreateService and UpdateService API. I am probably wrong, but these API calls

Runs and maintains a desired number of tasks

The key word is desired. I do not know what is my desired number because I haven't met the high load yet. When my containers gets hot on CPU/RAM/Bandwidth the desired number increases. Otherwise, the number of running TaskDefinitions should decrease.

Am I missing something?

You can start with a number that you think is reasonable, but using CloudWatch (for example) you can trigger the UpdateService API to increment the number of tasks based on the metric(s) you care about.

I have 2 concerns about ECS. 1) How do you ensure optimal instance sizing to ensure you arent paying for compute resources that containers arent yet using? 2) Say you stand up your services on a customer specific basis and you are running many customers. How can you ensure that ECS scales so that one set of services doesnt compromise the function of the rest of the services.

1. Two strategies here today. You can start with smaller instances (as long as they meet the minimum requirements of your applications) and autoscale. Alternatively you can be conservative and start with larger instances and monitor overall cluster utilization to keep the right number. We are working on providing you task and cluster-level metrics that can help you track cluster utilization: https://github.com/aaithal/amazon-ecs-agent/tree/telemetryacs

2. I think the metrics mention above will help you achieve that (Assuming each customer is their own task). We would also be interested in more discussion on this topic, so it would be awesome if you add this question to our forums: https://forums.aws.amazon.com/forum.jspa?forumID=187

Intentional that the GitHub link goes to a fork of ecs-agent rather than the source?

It's our fork under development

Awesome work team! I'm having a blast experimenting with ECS on elastic beanstalk.

Are there any plans to ease hosting private images, like the Google Container Registry? I've hacked together a solution using the S3-backed docker registry container, but I'd love to simply point to an S3 bucket as a "registry" and use IAM for auth.

Hi

Looks like this is a popular request. I'd definitely recommend dropping by the ECS forums (http://amzn.to/1PYrErc) so that we can have a deeper discussion and include other customers.

Currently, we are using ELB+ASG+EC2. It works great. We have a dozen of instances which automatically gets more than a dozen on high load. But we would like to migrate our VPC from spinning AMI's to spinning containers.

Please, recommend a strategy, step-by-step guide, or document to migrate from AMI's to ECS Docker Containers as smooth as possible. Is there one already?

Looks like this is an area we could improve our documentation based on the questions today. Use CreateService to launch services and then the UpdateService API to scale your tasks. Watch the AWS Compute Blog for best practices on scaling tasks.

use CreateService to launch services and then the UpdateService API to scale your tasks

Can UpdateService scale up/down the number of instances?

UpdateService only scales tasks. You can use Auto Scaling to scale your instances.

Howdy again! Are there any near-term plans for a single ELB to support mapping from a single ingress port (e.g. 443) to multiple service ports on multiple endpoints/containers (e.g. 6743, 4632, and 7322)? Thanks!

We've heard this request from other customers. We are talking to the ELB team about this. Their GM and I have coffee together on most mornings. I suspect he is amenable to me buying him coffee everyday.

I was trying out ECS the other day, but it looks like it doesn't work with our private registry running Docker Registry 2.0. Am I doing something wrong or does ECS just not support it?

Currently we support Docker 1.5. Docker Registry 2.0 is officially supported with Docker 1.6. We are qualifying Docker 1.6 as we speak.

Howdy! Are there any near-term plans to support running > 1 copies of a given container on a single instance w/o specifying them as additional task definitions? Thanks!

Howdy!

You can run multiple tasks from a single task definition on a single container instance as long as the resources are available.

Hm, not to devolve into support - we started a test service task with 5 requested copies against 3 container instances, and received only 3 service tasks. Could this limitation have been a result of something port mapping-related?

If the resources are not available and you are using host port binding that is the likely problem. Check your event stream on the service to see if that is indeed the case.

Are there plans to provide hosted orchestration layers for containers as a service (kubernetes, fleet) on top of ECS? Does Amazon intend to build their own?

Does our Service Scheduler (http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) work for you? It is designed for long running services.

What is the recommended approach for IAM permission use within Docker containers, any plans to have the ecs agent be able to forward into the containers ?

You can use IAM roles just like you do today (they are accessible from within your container). You should be aware that the IAM role is available to other containers running on the instance.

Is some cases this might be security issue. For instance I need have an access for one container to S3, but I have to give to all on this instance. Is there any way to prevent it?

Remember that you own the instances that your containers are running on and can set up the appropriate privileges. But it looks like you would be interested in per-container or per-task roles. If yes, we'd love to discuss this with you on the forums (https://forums.aws.amazon.com/forum.jspa?forumID=187)

json is great for software to exchange data between themselves, but it's terrible for human being: No tooling, no comments, commas hell (if you miss one, it invalidates the whole file), difficult to debug, no schema to test against etc...

Have you guys thought of adding another config language like yaml or even xml (hey, at least there's tooling for it)?

Hi,

YAML is definitely easier for humans to consume. I would recommend that you take a look at this Github repository: https://github.com/ambitioninc/container-transform that allows you to transform YAML-based Compose templates to ECS Task Definitions (and back)

When I created an ECS instance, I noticed that my full list of security groups was not available for selection. Instead, I had to manually add the desired rules into the assigned security group. Why is this?

We're interested in getting additional detail on this one. If you drop by the ECS forums (http://amzn.to/1PYrErc) we can definitely dig into this.

With bigger images, deploy feels like they take forever.

When will AWS have its own docker registry?

Thank you for the feature request. Could you please post this on the ECS forums (http://amzn.to/1PYrErc)? Your feedback definitely helps us prioritize features and allows other customers to weigh in.

Is there a best practice or planned support for local environments which use ECS for production/test workloads?

There was mention of eb local (elasticbeanstalk engine for local use) but that has yet to be released.

We'd love to ask you more questions about this. Could you post this question on the ECS forums (http://amzn.to/1PYrErc)?

Hi,

My two main issues with ECS as currently delivered are: - I think the EC2 security group per instance model is a poor fit for a container world where many containers run on the same instance. - I believe any networking model that provides less than a unique, routable IP per container is flawed.

Are any of these issues something you're looking to address?

Yes! :-)

Care to expand? I'm especially interested in what you believe is an ideal networking model for containers. Overlay network (VXLAN etc)?

The one that works!! But seriously, any answer to this question requires a lot more time than we have in this AMA. Even within our team there is more than one opinion on this question.

How much of what you get to work on is open source? Are you allowed to contribute back to upstream projects?

I'll answer for the ECS team. The ECS Agent https://github.com/aws/amazon-ecs-agent) and the ECS Init RPM (https://github.com/aws/amazon-ecs-init) are both on Github under OSS licenses. We've also contributed code to Docker (https://github.com/docker/docker/pull/5868)

Thanks for the AMA.

Can you ELI5 what ECS is and how someone with >200 ec2 instances with multiple environments can take advantage of it?

Say I have puppetmaster, ELB, frontends, backends, NFS, RDS, Cloudfront. Would ECS make my life easier?

On the first part of your question: ECS is ideal for people you run a number of instances, are running a mix of service-oriented and batch applications, and want to orchestrate them efficiently across a cluster.

On the second part, I am in the "yes" camp.

Howdy again! Any ETA on facilities to delete task definitions? Thanks!

The team is looking into this.

Are there plans for ECS to work with dedicated servers? Due to security concerns, we can only run our software on dedicated instances, so it would be awesome to have a cluster of dedicated instances with containers managed by ECS.

ECS will run on whatever EC2 instances you provide (we recommend instances running within a VPC). This includes dedicated instances. So you should be able to launch your own dedicated instances and use them with ECS. The key is to make sure you are running an AMI that supports Docker and is running the Amazon ECS agent.

We would like to use Amazon ECS, but we have Service-Oriented Architecture + Docker, so we have lots of internal APIs behind the scene, we have 3 environments (prod, test, dev), ~40 Services per env (foo-api, bar-api, foobar-api ...) so in current shape (ECS), we need to spend at least 18 USD per service with ELB * 40 services * 3 envs = 2160 USD per month only for ELB (not including traffic). Do you have any other plan for service discovery, not related to ELB? We really like AWS and we don't want to setup our own Mesos + Kubernetes at EC2 or move to GCloud but ECS IMHO is not SOA friendly. Can you give us some advices?

There are many options to do this today. You can use Eureka from Netflix (https://github.com/Netflix/eureka) or take a look at AirBnB Synapse and this pull request: https://github.com/airbnb/synapse/pull/110. The pull request allows linking a Synapse container with your application and the Synapse container with Haproxy will forward requests to the location in your cluster where the targeted services are running and dynamically reconfigure as services stop and start by using our Describe APIs.

Thanks so much for providing the platform. I've been using ECS for our dev stuffs in Ireland region but we can push to pre-prod because we currently use singapore region (ap-southeast-1) as our main dc, when will ECS be available in singapore region?

Thank you for your feedback. As with all our services we plan to expand to AWS regions and your feedback definitely helps us prioritize rollout.

Very excited for ECS! Any chance we can get support for zipped Dockerfile + context like on Elastic Beanstalk?

If I understand your question correctly, you will find this blog post on using Jenkins with ECS useful: https://blogs.aws.amazon.com/application-management/post/Tx32RHFZHXY6ME1/Set-up-a-build-pipeline-with-Jenkins-and-Amazon-ECS

For the uninitiated here, could you please briefly introduce the EC2 Container Service?

For my question: Can it benefit users of EC2 Spot Instances?

Thanks

TL;DR> https://www.youtube.com/watch?v=zBqjh61QcB4

Absolutely. We think of ECS as a way for you to efficiently run Docker-enabled applications across a cluster of EC2 instances. We focus on the complexity of state and cluster management, and let you focus on how you want to split up your applications into containers.

And yes, you can use Spot Instances with ECS.

I'm a layman and just stumbled in here.

What is Amazon ec2 container service?

https://www.youtube.com/watch?v=zBqjh61QcB4

How often do you all roll out changes to your Production service? What does your deployment process look like - do you roll out changes to a small # of customers, monitor for bad effects, and then continue with the rest of the Production environment?

Do you use any open source tools for your deployments, or is everything built inside of AWS?

I'm curious what patterns AWS use for their own services.

We are like many other Amazon teams. This talk will give you a sense of how we do deployments (https://www.youtube.com/watch?v=KzziRHOa5X4).

Long story short: We do a lot of deployments :).

What are your ringtones?

• Deepak: Led Zepellin's "Kashmir"
• Anthony: Whatever the default iOS ringtone is
• Sam: UCLA fight song
• Dan: AT&T Firefly
• Everyone else: Silent

I realize Amazon EC2 has nothing to do with Amazon Mechanical Turk... but do any of you know a guy who knows a guy that can get me the mythical Masters Certification?

I have no idea what that is, but I want it too :-)