Deploying Adaptive Apps in the cloud

At Platform Engineers Singapore on 19 October 2022, members joined in person and online to learn patterns and best practices for deploying and managing applications in the cloud.

Session 1: Adaptive Apps, a new pattern for cloud application management

Adaptive Apps is a new design pattern concept. Application focused but serving infrastructure, connectivity, management and observability all in one. Speaker: Buu Lam, Community Evangelist, F5 DevCentral https://www.linkedin.com/in/buulam/

Technical Debt is where you inherit, or develop over time, a complicated technology environment that holds you back from delivering new services and adapting to changing customer needs. Every technical decision involves trade-offs, such as just building something for the short term, or lack of budget or resources. The trouble is that these trade-offs and resulting compromises can build up to result in a technology landscape that is now difficult to manage.

This increases security risk, because of all of the components and the technical debt that you've inherited result in configurations that need to be kept up to date, and risk of misconfiguration of security and access.

It also increases operational complexity. Operational costs are higher, and it takes longer to deliver new functionality due to having more systems that need to be updated to implement a change. Adding to that is the problem of knowledge management - your team developing an understanding of this complicated environment.

So at F5, we've developed an architectural framework that we call adaptive applications. We have a lot of pieces that deliver an application. This framework seeks to address security threats, performance, and resilience, speed of deployments, for new applications, unifying policy across environments as well.

This is becoming really important when we're looking at multi cloud worlds. It's complicated to be able to manage a singular policy that applies to different types of clouds that have all their different ways of deploying. So this architecture is broken up in two ways.

  1. We have a services tier for global shared services. There's certain parts of your application delivery that are going to sit in this global services tier. You could think about it physically, or you could think about it logically, because some of these mobile services could just exist as a SaaS service. And then there's specific services that are shared, such as sites, so that could be firewalls or networking. And then we have the app services tier. So specific components that are required for those applications. We break that up, as well. And then we have this management and operational services here. It could very well live inside of a cloud provider, but the important thing is to move it off the data plane, onto a separate control plane.

  2. Unified Security and connectivity. The idea here is to design unified policy definition enforcement. So we want to be able to say, for example, for AWS and Azure, we're going to have the security policy network security policy. And regardless of how it's deployed within there, we're going to have that singular policy. And then the individual API's will take care of that, at that point, there's, there's going to be a point where it joins together, system telemetry and advanced insight. This is one where we need that over in the management and operational services. And from that, we should be able to gather enough information to start making smart decisions about how to handle things that are going on in the environment such as reacting to services going down or be able to being able to scale. So that would fit into their common tooling and automation.

Having all that data being fed back into the system enables self healing. Also, when we are able to globally define policies, network policies, security policies, connectivity policies, then when the application developer actually wants to push down an application, it just inherits all of that.

Watch the recording of Buu's presentation

Watch Buu's full presentation here:

Session 2: Smart City in the cloud

JTC's Open Digital Platform integrates Smart City solutions. Topics:

  • Pros and cons of cloud solutions
  • When to go IaaS when PaaS
  • How Open Digital Platform is using both
  • Dive in to PaaS solutions e.g API management

Speaker: Celine Chia, Senior Software Engineer, GovTech & JTC, https://www.linkedin.com/in/celine-chia-22110404/

We formed the Smart Division Program Office at Singapore's Government Technology Agency (GovTech). Together with JTC, we drew up the first map district in Singapore, the Pungol Digital District. The Pungol Digital District is expected to be commissioned in 2023 to 2024. While building this up, we have already commissioned one of the main smart buildings, which is at JTC Summit. So that is our testbed. And we have already commissioned it in May this year. So this will form our baseline and we will periodically go into Pungol Digital District to deploy whatever that we have already built in.

So we are using cloud services to accelerate our development and delivery process. We have a small team, and to actually to get the entire smart district ourselves from scratch is entirely impossible. So a cloud solution is something that we always go for whenever we can.

In order to put together the entire smart district city together, we need people with a wide variety of skill sets: infrastructure, IoT, AI ML product and modeling.

So today's agenda I want to share with you is the pros and cons of cloud versus on-premise solutions. What are Infrastructure-as-a-Service (IaaS), Platform-as-a-Service and Software-as-a-Service. And this Open Digital Platform that GovTech and JTC we have built together. I will give you an example of how we are using our PaaS services, the API Management.

I will go deeper into what we are doing in in terms of the Smart City. The Pungol Digital District is is strategic national project, the first smart district that we have in Singapore. We are building the Open Digital Platform (ODP), which is actually a smart district OS. So because this this district will house high technology jobs to potentially create 28,000 jobs. And because it's a smart district, it's important to have the academia element, so SIT (Singapore Institute of Technology) is within this. Already there's a lot of smart systems integrated in. By itself smart systems are diverse: smart lighting, smart toilet, etc. What they do already do is really smart, but they are usually in silos. To actually make the entire street or system smarter, we need to build the open the protocol middleware. So we want to enable the correlation of the data sharing to enhance it and make it easier. What ODP does is we are building the the open standard multi protocol middleware. This multi protocol middleware will allow different kind of protocols to enter into the same one horizontal platform: MQTT, API, AMQP will just be ingested into this one layer to allow easy access of data sharing. On top of that we are building a digital twin.

Watch the recording of Celine's presentation

Watch Celine's full presentation here:

Session 3: Speed developer onboarding with an Internal Developer Portal

An Internal Developer Portal (IDP) promotes best practices by showing patterns and documentation so that developers can find and follow the steps to build and deploy applications easily. Speaker: Jon Scheele, Founder & CEO @ Blue Connector, Organiser @ Platform Engineers Singapore and apidays Singapore, https://www.linkedin.com/in/scheelejon

When a new developer or engineer comes onto your team, you have this problem or how you transfer knowledge to them so they become productive as soon as possible. I will share about knowledge sharing and documentation, and how you can accelerate this with an internal developer portal. Some of this is inspired by a book I've called "Software Engineering at Google", which has chapters on both knowledge sharing and documentation. For the internal developer portal, I'm going to introduce you to a tool called Backstage.

Firstly, knowledge sharing: Why do we have challenges with learning new things, in particular all things we need to know about what is going on in an organization? One reason is that people don't want to ask questions that appear foolish, they don't want to take a risk, because they may make a mistake and get blamed for something going wrong. Knowledge also exists in pockets across an organization. You have people who have developed expertise over a long period of time, but new people coming on board don't know where to find knowledge or the people who have it. This fragmentation leads to duplication of information. Experts can become the go to person for a given subject, but others may get comfortable in always being able to ask the expert, so they don't bother to learn themselves. So you get these situations where somebody either knows a lot about something or they know very little.

Knowledge takes different forms. It can be personalized one-on-one, which is the most powerful, but doesn't scale very well. Documenting that knowledge is scalable, but it tends to be general, not specific to a particular person's problem. Then there's the tribal knowledge, which is those things that aren't written down, but generally known across the across the team.

You can scale your knowledge with group chats, mailing lists, a Q&A platform (like Stack Overflow), but it can be hard to manage that information or to find information that you're specifically looking for. Sometimes the issue is just so complicated, that you need to find an expert to explain it to you. And that's where they're less scalable things like office hours and tech talks and can come in.

Documentation should explain not just how something works, but why certain decisions were made. The person who designed that system had to make some trade-offs. Do these still apply?

There are several benefits of good documentation:

  1. The simple act of writing something down helps you to clarify thoughts in your mind as to how it can work.
  2. It provides a record of what works and what doesn't.
  3. It makes you look more professional.
  4. If your documentation looks good, then people will look at that and say, "Oh, that's a good system, I'm going to use that".
  5. It can reduce the amount of questions you get asked about your code.

So you should put as much effort into your documentation as you do into the code.

A developer portal benefits several roles:

  1. Engineering managers: they're really keen on maintaining standards across the organization, so if somebody has figured out the best practice, they want to be able to have the entire team follow that best practice and not a multitude of different ways.
  2. Developers: it should help you to create a new environment that incorporates all those best practices easily.
  3. Platform engineers: want to know where everything is, and make sure that they can bring in new tools and incorporate them into the whole platform.

Backstage (https://backstage.io) as a developer portal has become popular is because it incorporates not just static documents, but also gives that view into how you can create a new environment. It was developed by Spotify, then open sourced, and is now under the Cloud Native Computing Foundation. It contains:

  1. A software catalog, listing all your all your API's micro services, models, and so on.
  2. Templates for how you can create a new instance of an application. If you're building a node.js project, and the organization you're with wants things done a certain way, with certain components to it, and certain styles, you can encourage that by creating a template that people can use to replicate environments.
  3. Documents
  4. Plugins: Backstage can be extended with plugins for new components that the people who built Backstage didn't think of.

Switch to demo...

Watch the recording of Jon's presentation

Watch Jon's full presentation here: