Archive

Archive for the ‘Cloud’ Category

Connecting VMware Cloud on AWS to Native AWS Services – a UKVMUG presentation

December 13th, 2018 No comments

I had the huge pleasure today of presenting a community talk at the UK VMUG in the stellar location of the UK National Space Center in Leicester…there was an actual rocket!

20181212_185857148_iOS 20181212_190135733_iOS

20181212_190547662_iOS 20181212_190133200_iOS

I’ve been super interested in the recent love fest of VMware and AWS. The titans of private and public clouds have entered into a marriage of stratospheric nerdy potential. I recently attended AWS re:Invent with plenty of blogging! What interests me more than just running vSphere VMs on ESXi on Amazon hardware (which is a fancy colo if you ask me) is being able to take advantage of all the myrid AWS services from within those vSphere VMs. AWS has a vast array of service which can be consumed directly and can really help augment your tried and trusted vSphere VMs.

Presentation TL;DR

I went through a short overview of the VMware Cloud on AWS service and delved more deeply into the key networking component which is the Elastic Network Interface. This is an AWS virtual Nic which connects the two clouds together. Then onto some real use cases like connecting vSphere VMs to storage with S3/EFS, connecting an app running on a vSphere VMs to a RDS database running in AWS. Then turning the connectivity around and using an AWS load-balancer to front a connection to a pool of vSphere web servers.

I then spent some time going into the up and coming AWS-VMware options including RDS on vSphere which is running AWS RDS databases on vSphere in your datacenter. Then I also spoke about the very recently announced AWS Outposts which is AWSs surprising hybrid cloud announcement. Who would have thought but AWS hardware on-prem which has an option to run a vMware SDDC. There’s also CloudFoundation for EC2 which is using vSphere management and networking for EC2 instances wherever they may now be able to reside.

Thanks for attending for the people who were there and if you weren’t able the be here today, hopefully the slides are useful.

I particularly need to make special mention of the screen shots I borrowed from many other VMware and AWS presenters to make this presentation. I don’t actually have a VMware Cloud on AWS in my back pocket so needed to lean on the creation of others and curate their hard work for today’s presentation. See the slides for who helped.

Here are the slides:

AWS re:Invent 2018: The Day 3 Buzz

November 29th, 2018 No comments

20181127_004442894_iOS

Into Day 3 of the conference which is when more of the announcements start rolling in!


Andy Jassy Keynote

As I’ve settled into Vegas time zone, this felt like an early wake up to make it in time to the keynote as there was going to be a queue. It would be far more sensible to stay in a hotel or other venue and watch remotely but feeling the reactions in the room for the announcements seems more interesting.

The keynote was streamed to some big spaces in the other conference venues and a good change this year was also streaming it to many more of the breakout sessions rooms all across the venues so you had more chance of seeing the keynote without queuing like crazy.

AWS has made so many announcements in the build-up to re:Invent one wonders whether they’re trying to hit a particular release number to flash onto the big screen! A quick way to see the list of announcements is to look at AWS What’s New 2018:

CEO Andy Jassy as usual was master of disclosure.

I had no intention of live blogging the keynote, far too much information and others who are quicker typists!

There was a DELUGE of announcements, some recaps from the few weeks and many new…I needed to take stock a few times, pause and try to make sense of it all.

I blogged about separately: AWS re:Invent 2018: Andy Jassey Keynote Thoughts

Read more…

Maybe spend time looking at how new tech CAN help you rather than CAN’T help you

September 12th, 2018 No comments

I was recently invited to do an internal enterprise financial company presentation on serverless computing as part of a general what’s happening in IT series. There was a wider range of people than I expected who attended, some business people and some IT people.

The business lens

In the questions and feedback afterwards interestingly some of the business people could see some of the value more easily than the IT people. Business people liked the coming together of business logic and IT and could see the benefit of just encoding what they need doing in a serverless function without having to worry as much about all the IT infrastructure stuff behind the scenes. Although the business people weren’t coders, someone likened the approach to using Excel macros. Some fairly sophisticated Excel functions have graced the trading desks of many an organisation. They didn’t need to think about infrastructure with Excel Macros, Excel was just a platform you could code mathematical functions in. Sure, Excel macros had many issues, security, performance, availability etc. but they served the business need easily without having to get IT involved.

The IT lens

I then spoke to a development team leader afterwards. She’s very well versed in coding, a super smart algorithmic trading developer. She voiced valid concerns though that with serverless functions you couldn’t control the latency of the function and so she couldn’t see any use for them in their work. Part of the workflow they develop is low-latency trading, pricing and analytics which of course is very latency and performance sensitive. Some of the workflows include many steps necessary for compliance and auditing. A price range traded may need to be put into a database to reference later. A trade that is priced needs to be logged somewhere and a trade completed needs to go into another database which kicks off a whole other bunch of workflows to be reconciled in the back-office. She mentioned the low-latency algo stuff was working well but they sometimes struggled with performance and speed when it was a very busy trading day. Some of the compliance and auditing code sits very close compute wise to the low-latency code. This makes it simpler to code the end-to-end transactions but it means the most expensive physical server hardware low-latency compute cycles are also being “wasted” on compliance and auditing code which may struggle to keep up on an extra busy trading day. To improve this would generally require scaling up existing compute resources. The compliance and auditing data was also used by many other integrated systems so care needed to be taken so that the secondary databases could keep up with low-latency demand.

This made me think of two things, first of all how this application would of course benefit from some splitting up. The app could be changed for the low-latency code to push out the minimal amount of compliance and auditing information to another database, queue or even stream. A separate set of serverless functions could then very efficiently respond to an event or pick up these trades or prices and do whatever needs to be done (BTW, its not just functions that can be serverless, databases, queues and storage can be too!). This could also be massively scalable in parallel. 1 trade at a time or a million and this wasn’t latency sensitive stuff once the initial small record was created.

CAN use or CAN’t USE

Secondly was how the developer team leader was seeing how serverless functions COULD NOT be used for latency sensitive workloads but not seeing how useful they COULD be for all the rest of the compliance and auditing code. The low latency code was the most important so naturally her focus is on that.

The splitting up of the app is an architectural discussion and may not in fact be suitable in the end but the more important point is sometimes we are a little myopic and only see what a technology CAN’T do rather than looking at the bigger picture and seeing what it CAN do. This can distance you from the business. Oh, and of course, Excel can do a LOT!

Categories: Cloud, DevOps, Scale, Serverless Tags:

AWS re:Invent 2017: The Show Review

December 19th, 2017 No comments
Categories: AWS, Cloud, re:Invent, Serverless Tags: , ,

AWS re:Invent 2017: What’s New in Serverless – SRV305

December 1st, 2017 No comments

Tim Wagner the AWS Serverless GM and Jeet Kaul from FICO

This session was about new things in serverless.

Tim reiterated how mazing Lambda is coming, its even inside a camera which was announced in the keynote as well as the top memory size being doubled to 3Gb which also doubled the CPU power.

2017-11-30 15.18.52Magic!

There was a mini magic show which was apparently a nod to something they did last year.

The idea is to show disappearing servers, as there are more and more serverless offerings, this means more and more disappearing servers.

Serverless Application Repository.

It’s worth looking at the recently announces Serverless Application Repository, its a marketplace of serverless functions published by AWS and others.

Read more…

AWS re:Invent 2017: How AWS Runs Our Weekly Operations Meetings – ENT346

November 29th, 2017 No comments

2017-11-29 14.29.54David Lubell and Kevin Miller from AWS

I was really looking forward to this session, as this is the very first time AWS has opened the kimono on how it actually runs its own operations.

I was there two hours in advance to guarantee a place, they only had a room for 60!

David started off by mentioning that AWS has had a weekly operational meeting which has now been running for more than 10 years. It runs for 2 hours every week. This looks at the  performance of services with the idea of nipping issues in the bud as well as being forward looking by identifying new best practices.

David wanted to share some lessons learned from what he termed “the trenches” with one of the largest services in the world.

In every meeting, leaders for every AWS service together with more than 100 engineers deep dive into their operations. The reason for having so many people is to have immediate ownership of things and be able to more quickly respond across the whole organisation.

David went through ways they’ve developed to run an effective meeting which is so large. It’s not just about the tech they’re looking at but also how you can run a meeting to look at this effectively, how can you feed back the metrics you can see to stop issues recurring. AWS understandably has a very high bar for operational performance.

In the meeting, they go through and share successes, look at organisation projects, review operational events, do service metrics reviews and other updates and announcements. It’s not just about metrics but is an ops information sharing forum which sends the right message that operations really matters. It helps to gather a community and tech best practices with an accountable audit mechanism. They try to avoid spending too much time on things specific to each team.

Each team has a standard set of metrics (responsiveness etc.) and also metrics for what’s special about the service. Customer service always important so they measure how long it takes an EC2 instance to get fully up and running for example as well as how long the console displays the updates.

Read more…

Categories: AWS, Cloud, re:Invent Tags: , ,

AWS re:Invent 2017: Tuesday Night Live with Peter Desantis

November 29th, 2017 No comments

Tuesday Night Live Peter Desantis

2017-11-28 20.14.21 2017-11-28 19.40.26_thumb[2]

Peter is VP of AWS Global Infrastructure which he’s been doing for a year and a bit although he’s been at Amazon for 20 years. This slot has always traditionally been done by James Hamilton, AWS Distinguished Engineer & one of the super techies.

Talk on the street is that all the techie stuff they are likely to talk about in this session is actually the stuff AWS was using two years ago and now wants to talk about, they’re so far ahead that they keep some of their new cards close to their chest.

2017-11-28 19.11.17There was a cool warm up band.

Update on AWS Global Infrastructure

Write your app once and deploy to any region “going global has never been so easy”, said Peter.

He went through the history of the global expansion and how this is accelerating. They have announced plans to expand with 17 new Availability Zones in six new geographic Regions: Bahrain, China, France, Hong Kong, Sweden, and a second AWS GovCloud Region in the US.”

Just let that sink in, 11 regions in the first 10 years and 17 in two years, that;s massive scale.

He touted the AWS commitment for renewal energy. He continued drilling down into what regions look like with the make up of availability zones being separately powered and super redundently cabled together.

Machine Learning

Read more…

Categories: AWS, Cloud, re:Invent Tags: , ,

AWS re:Invent 2017: Scaling Up to Your First 10 Million Users – ARC201

November 28th, 2017 No comments

Benjamin Thurgood from AWS

Cloud is awesome, you can scale your web app or site on demand! But where do you start when you’re planning for bi g numbers of users. Making some design choices up front can dramatically simplify your life later.

Benjamin showed how best to link together different AWS services and using the capabilities of the AWS platform.

Autoscaling is only the beginning, there’s a lot more to look at.

Use Native Sevices

First step is to use the AWS Global Infrastructure with Edge locations. Take advantage of the global AWS platfom so you don’t have to reinvent the wheel.

Many native features available like Cloudfront, S3, DynamoDB, EFS etc.

EC2, EBS, RDS etc are scalable but you need to work a little harder.

Lightsail is an easy way to get started as a developer.

You can scale up but you’ll reach an end some time.

Read more…

Categories: AWS, Cloud, re:Invent Tags: , , , ,

AWS re:Invent 2017: Build a Multi-Region Serverless Application for Resilience and High Availability Workshop

November 28th, 2017 No comments

Steven Challis & Derek Felska from AWS were the workshop leaders and it was very hands on, basically up to you and anyone else you wanted to team up with.

2017-11-27 12.58.36 2017-11-27 12.58.45

This is one of the reasons to actually attend a conference, you get to do things in person and interact with other people rather than watching a recorded session or just follow a step by step plan when you can’t confer.

Intro

Availability and fast performance is key to user experience. Building a global application from the start is traditionally extremely difficult. Think before serverless how you would have to manage a global fleet of EC2 instances, load balancers, databases and storage. You would need to be a DNS guru and keeping your compute generic yet regionalised was super tough. Enter serverless and the promise was there but Lambda needed a whole lot of hacking to get functions to fire based on geographical access.

2017-11-27 13.15.14 2017-11-27 13.05.29

In the workshop we set up a fictional company called www.wildrydes.com (would you use a ride sharing company called this!). This wasn’t just a normal rider sharing company though, the drivers were unicorns! They needed a customer support application which customers can use to report any issues, be it lost property or a grumpy unicorn! As the service was global and needed to be built, serverless was touted as the ideal platform to use as much as possible (of course, it’s re:Invent!). We needed to lash together Lambda, API Gateway, DynamoDB, Route 53, CloudFront and S3 for better availability. Cognito Federated Identities was also used for user authentication.

The workshop was also to highlight the new “API Gateway regional endpoints” feature which was recently released.

There’s no reason to feel left out though, you can go through it all at: https://github.com/awslabs/aws-serverless-workshops/

Read more…

AWS re:Invent 2017: Security Anti-Patterns: Mistakes to Avoid – FSV301

November 27th, 2017 No comments

FSV301 – Security Anti-Patterns: Mistakes to Avoid

Kurt Gray from AWS and Jonathan Baulch from Fidelity Investments

AWS has obviously spent a huge amount of effort building security into the very fabric of its cloud offerings. Enterprises still hesitant to use public cloud as they’re concerned about the security implications of AWS in my opinion are thinking old school. They are often taking the processes and procedures that they currently use on-prem and applying it to AWS rather than looking at all the new possibilities which are often more secure than they can do themselves.

AWS famously has touted the security split of “security OF the cloud” which is AWSs job and “security IN the cloud” which is the customer’s responsibility, however AWS has a huge number of tools to help with the IN part.

Kurt and Jonathan went through some of their learning about best practices and pitfalls. There are a number of governance patterns to avoid even though they may seem logical at first but may limit scale and throttle getting stuff done. This is all bundled under the banner of DevSecOps on a massive scale.

Read more…

Categories: AWS, Cloud, re:Invent Tags: , ,