The Carbon Monkey

How principles of Chaos Engineering and using carbon monkeys to simulate real-life energy events help us achieve our sustainable software engineering goals.

Photo by Singkham on Pexels.com

According to Principles of chaos engineering, Chaos Engineering is the discipline of experimenting on a system in order to build confidence in that system’s capability to withstand turbulent conditions in production. I have followed this discipline through the years finding it fascinating, especially when applied to large scale applications and systems. As the site explains:

“Even when all of the individual services in a distributed system are functioning properly, the interactions between those services can cause unpredictable outcomes. Unpredictable outcomes, compounded by rare but disruptive real-world events that affect production environments, make these distributed systems inherently chaotic.

We need to identify weaknesses before they manifest in system-wide, aberrant behaviors. Systemic weaknesses could take the form of improper fallback settings when a service is unavailable; retry storms from improperly tuned timeouts; outages when a downstream dependency receives too much traffic; cascading failures when a single point of failure crashes; etc. We must address the most significant weaknesses proactively, before they affect our customers in production.

We need a way to manage the chaos inherent in these systems, take advantage of increasing flexibility and velocity, and have confidence in our production deployments despite the complexity that they represent. An empirical, systems-based approach addresses the chaos in distributed systems at scale and builds confidence in the ability of those systems to withstand realistic conditions. We learn about the behavior of a distributed system by observing it during a controlled experiment. We call this Chaos Engineering.”

Build a Hypothesis around Steady State Behavior

Let’s start with the first step: a steady state behavior is the condition our application should aspire to be in. If we translate this principle into a sustainable one, this becomes the most beautiful and efficient state of an application: one where no energy is wasted, and efficiency and performance is at its best.

Call for more “carbon monkeys”

The most difficult part is how to measure and set this initial state. My colleagues have shared numerous ideas on the Sustainable Software Engineering blog that might help you jumpstart your measurement. However, I feel that at some point, this will have to reach a standardized and widely accepted form where we have a “carbon limit” where an application is considered inefficient and not sustainable.

Vary Real-world Events

This is the principle that represents how close chaos engineering and sustainable software engineering are. There is no steady and predictable flow of energy coming from the same renewable source. From the challenging big picture of using solar, wind or hydro energy down to when we plug our device into the outlet, we still have limited ways to retrieve exactly how the energy that is powering the device is produced in that exact moment in time. Doing so precisely requires considering things like seasonality, time of day, peak hours as well as weather conditions that trigger renewable power supplies usage. The variables around this concept are too many!

Imagine now that your application is running on a virtual datacenter where you have even less information of its carbon impact. We still need to start somewhere, though, and set an amount of carbon usage for the application. This will be useful to measure its increase and decrease to drive efficiency.

Back to chaos engineering. Simulating power outages is just a start. We can think of it as the starting point for a sustainable application:

  • What if the renewable power sources are suddenly unavailable and therefore, I have spikes of energy consumption that I could not foresee even in the greenest application?
  • What if at some point my application has become a “carbon monster,” greedy with energy because a query has gone wrong and it’s suddenly taking most of its energy just to search for that item in your cart? Or because at some point the network path has changed due to an outage in the network route and its latency spikes? And so, trying to replicate real-life energy events directly into an application will make it more resilient to lower energy availability and overall, more efficient.

Enter the “Carbon Monkey”

This concept is a “carbon” monkey: a process or system that triggers energy inefficiencies at random, testing how your application reacts, and measuring differential performance that can relate to the differential carbon impact.

Instead of measuring how much energy an application consumes, we should test adding energy events to see how the application behaves and then drive change to improve its reaction to events that make it less green. 

We have given the problem of how to measure an application’s carbon efficiency a lot of thought. But this approach offers a change of perspective. Instead of measuring how much energy an application consumes, we should test adding energy events to see how the application behaves and then drive change to improve its reaction to events that make it less green. 

As a result, we won’t have a carbon impact exact measurement, but only a differential. With time, this differential can become an absolute number when  other systems allow us to retrieve more precise energy consumption metrics.  In the meanwhile, let the carbon monkey help us reduce impact regardless of the metric standardization!

Photo by Alexandr Podvalny on Pexels.com

Call for more “Carbon Monkeys”

I’d like to see developer communities creating one or more “carbon monkeys” that can introduce energy-impacting events into applications, to foster resiliency towards sustainability. 

The main trigger is defining a set of incorrect assumptions about energy usage that can prevent our application from performing “green”. These would include assumptions such as the highest energy cost/carbon use/region, the shortest/longest queries, the shortest/longest network paths, the highest compute and memory usage among other things. 

These assumptions should then be introduced by an automated process (our monkey) that will make sure that the application patterns are resilient enough to overcome those issues without completely failing. At the end of the run, we could set up a carbon resiliency value that can help set a standard for the application carbon impact differential evaluation.

Originally published in the Microsoft Developer Blogs

Sustainable cloud native software with serverless architectures

Living in Milan, I have had to deal with extraordinary air pollution values since December 2019, in some days controversial graphs compared Milan to much more densely populated and polluted cities, in China and India, at least by common perception.

Then came covid-19, and obviously our concerns moved elsewhere. Like everyone, at least in Lombardy, I was in lockdown from 21 February to 4 May. In the midst of a thousand worries, a little voice in the back of my head continued to point out that, however, suddenly, the air was no longer polluted, the CO2 levels had dropped significantly, which in short meant that an important change and with impactful results was, indeed, possible.

Fast forward to now … do we want to go back to the impossibly polluted air of January 2020? If the answer is no, then something needs to change.

First, let’s see why a change is due and important. The whole scientific community agrees that the world has a pollution problem. Carbon dioxide in our atmosphere has created a layer of gas that traps heat and changes the earth’s climate. Earth’s temperature has risen by more than one degree centigrade since the industrial revolution of the 1700s.

If we don’t stop this global warming process, scientists tell us that the results will be catastrophic:

  • Further increase in temperature
  • Extreme weather conditions, drought, fires (remember the Australian situation at the beginning of the year?)
  • The rising of the waters could make areas where more than two hundred million people live uninhabitable
  • The drought will necessarily lead to a food shortage, which can impact over 1 billion people.

To summarize, we must drastically reduce CO2 emissions and prevent the temperature from rising above 1.5°C.

Problem. Every year the world produces and releases more than 50 billion gas into the atmosphere.

CO2 emissions are classified into three categories:

Scope 1 – direct emissions created by our activities.

Scope 2 – indirect emissions that come from the production of electricity or heat, such as traditional energy sources that power and heat our homes or company offices.

Scope 3 – indirect emissions that come from all other daily activities. For a company, these sources are several and must include the entire supply chain, the materials used, the travel of its employees, the entire production cycle.

When we speak of “carbon efficiency” we know that greenhouse gases are not made up only of carbon dioxide, and they do not all have the same impact on the environment. For example, 1 ton of methane has the same heating effect as 80 tons of carbon dioxide, therefore the convention used is to normalize everything to the CO2-equivalent measure.

International climate agreements have ratified to reduce “carbon” pollution and stabilize the temperature at a 1.5°C increase by 2100.

Second problem. The increase in temperature does not depend on the rate at which we emit carbon, but on the total quantity present in the atmosphere. To stop the rise in temperature, we must therefore avoid adding to the existing, or, as they say, reaching the zero-emission target. Of course, to continue living on earth, this means that for every gram of carbon emitted, we must subtract as much.

Solution to both problems: emissions must be reduced by 45% by 2030, and zero emissions by 2050.

Let’s now talk about what happens with datacenters, and in this specific case,  public cloud datacenters.

  • The demand for compute power is growing faster than ever.
  • Some estimates indicate that data center energy consumption will account for no less than a fifth of global electricity by 2025.
  • A server/VM operates on average at 20-25% of its processing capacity, while consuming a lot of unused energy.
  • On the other hand, in an instance where applications are run using physical hardware, it is still necessary to keep servers running and use resources regardless of whether an application is running or not.
  • Containers have a higher density and can bring a server/VM up to 60% of use of compute capacity.
  • Ultimately, it is estimated that 75-80% of the world’s server capacity is just sitting idle.

While browsing for solutions, I found very little documentation and formal statements about sustainable software engineering. While talking to fellow Microsoft colleague Asim Hussain, I found out that there is a “green-software” movement, which started with the principles.green website, where a community of developers and advocates is trying to create guidelines for writing environmentally sustainable code, so that the applications we work with every day are not only efficient and fast, but also economic and with an eye to the environment. The eight principles are:

  1. Carbon. First, the first step is to have the environmental efficiency of an application as a general target. It seems trivial but to date there is not much documentation about it in computer textbooks or websites.
  2. Electricity. Most of the electricity is produced from fossil fuels and is responsible for 49% of the CO2 emitted into the atmosphere. All software consumes electricity to run, from the app on the smartphone to the machine learning models that run in the cloud data centers. Developers generally don’t have to worry about these things: the part of electricity consumption is usually defined as “someone else’s problem”. But a sustainable application must take charge of the electricity consumed and be designed to consume as little as possible.
  3. Carbon intensity. The carbon intensity is the measure of how many CO2equivalent emissions are produced per kilowatt-hour of electricity consumed. Electricity is produced from a variety of sources each with different emissions, in different places and at different times of the day, and most of all, when it is produced in excess, we have no way of storing it. We have clean sources as wind, solar, hydroelectric, but other sources such as power plants have different degrees of emissions depending on the material used to produce energy. If we could connect the computer directly to a wind farm, the computer would have a zero-carbon intensity. Instead we connect it to the power outlet, which receives energy from different sources and therefore we must digest the fact that our carbon intensity is still always a number greater than zero.
  4. Embedded or embodied carbon is the amount of pollution emitted during the creation and disposal of a device. So efficient applications that run on older hardware also have an impact on emissions.
  5. Energy Proportionality. The maximum rate of server utilization must always be the primary objective. In general, in the public cloud this also equates to cost optimization. The most efficient approach is to run an application on as few servers as possible and with the highest utilization rate.
  6. Networking. Reducing the amount of data and the distance it must travel across the network also has its impact on the environment. Optimizing the route of network packages is as important as reducing the use of the servers. Networking emissions depend on many variables: the distance crossed, the number of hops between network devices, the efficiency of the devices, the carbon intensity of the region where and when the data is transmitted.
  7. Demand shifting and demand shaping. Instead of designing the offer based on demand, a green application draws demand based on the energy supply. Demand shifting involves moving some workloads to regions and at times with lower carbon intensity. Demand shaping, on the other hand, involves separating the workloads so that they are independently scalable, and prioritizing them to support the features based on energy consumption. When the energy supply is low, therefore the carbon intensity is at that time higher than a specific threshold, the application reduces the number of features to a minimum, keeping the essential. Users can also be involved in the choice by presenting the “green” option with a minimum set of features.
  8. Monitoring and optimization. Energy efficiency must be measured in all parts of the application to understand how to optimize it. Does it make any sense to spend two weeks reducing network communication by a few megabytes when a db query has ten times the impact on emissions?

The principles are generic for any type of application and architecture, but what about serverless?

Serverless applications are natively prone to the optimization of emissions. Since the same application at different times consumes differently depending on the place of execution, demand shifting is a technique that can be easily applied to serverless architectures. Of course, with serverless we have no control over the infrastructure used, we must trust that cloud providers want to use their servers at 100% capacity. 😊

Cost optimization is generally also an indication of sustainability, and with serverless, we can have a direct impact on execution times, on the network data transport, and in general on building efficient applications not only in terms of times and costs, but also of emissions.

The use of serverless brings measurable benefits:

  • The use of the serverless allows for a more efficient use of the underlying servers, because they are managed in shared mode by the cloud providers, and built for an efficient use of energy for optimal data center temperature and power.
  • In general, cloud datacenters have strict rules and often have ambitious targets for emissions (for instance, Microsoft recently declared its will as a company to become carbon negative by 2030). Making the best use of the most optimized resources of a public cloud provider implicitly means optimizing the emissions of your application.
  • Since serverless only uses on-demand resources, the server density is the highest possible.
  • Serverless workloads are ready for demand-shifting / shaping executions.
  • From a purely theoretical point of view, writing optimized and efficient code is always a good rule of thumb, regardless of the purpose for which you do it 😊

Developers can immediately have an IMPACT on application sustainability:

  • By making a program more accessible to older computers.
  • By writing code that exchanges less data, has a better user experience and is more environmentally friendly.
  • If two or more microservices are highly coupled, by considering co-locating them to reduce network congestion and latency.
  • By considering running resource-intensive microservices in a region with less carbon intensity.
  • By optimizing the database and how data is stored, thus reducing the energy to run the database and reducing idle times, pending completion of queries.
  • In many cases, web applications are designed by default with very low latency expectations: a response to a request should occur immediately or as soon as possible. However, this may limit the sustainability options. By evaluating how the application is used and whether latency limits can be eased in some areas, reducing further emissions can be possible.

In conclusion, I am convinced that serverless architectures, where properly used, are the future not only because they are beautiful, practical and inexpensive, but also because they are the developer tools that today have the least impact on emissions. With the help of the community, we can create specific guidelines for the serverless and maybe even an “carbon meter” of our serverless application, which in the future could also become “low-carbon certified”.

COVID-19 was an inspiring moment in terms of what we managed to do on a global level: all the countries stopped, all the flights, the traffic, the non-essential production. We know that something can be done and that this is the right time to act: rebuilding everything from scratch, it is worth rebuilding in the right direction.

AI and CX?

From Mobile World Congress 2016 to the recent F8 ten year roadmap speech, AI is definitely one of the hottest technology trends. And specifically, AI in the customer experience, which  is the front-line of any expectation towards a company has been buzzing for a while as an innovation topic.

News from the several AI experiments are not much reassuring: Tay’s Meltdown proved once again what my university professor would say of computer science, “garbage in- garbage out”.

So on one side we would love to have computers help us with our CX, but on the other it looks really risky as any AI exposed to the public can be manipulated to reflect bad, racist or inappropriate responses to apparently innocent questions, sometimes just for the sake of it, others because of a specific sabotage schemed to bring it down.

Within Customer Experience, the relationship with automation has always been controversial. Would a customer like to be served by a robot and to what extent? Why would a company want to invest significant amounts of time and money to expose its front-line and most visible asset to malpractice and gruesome attacks from trolls and hackers?

The problem of any AI is, obviously, the learning. So probably the mistake from Microsoft was to trust the public network to be truthful and honest when teaching conversational skills to its bot.

Having worked in the customer experience realm for many years, I would never trust a bot to learn from public behaviour over social media: imagine your new hire agent sent to learn conversational skills and empathy…in the street?

but….on the other hand, I know that these guys (the CX teams) are literally sitting on a pile of interaction recordings that are rarely used, unless for some sparse quality management or compliance regulation. So why not use this big data to teach an AI, in a controlled, business-like though still real-life environment, how a conversation about your own brand or product should evolve? This idea might not be new but I haven’t seen anyone even testing it yet. Probably the biggest refrain is that AI projects are still in an experimental phase, are very expensive and bring little certainty of results.

But think about this: if you could have your new hire listen to thousands of hours of work conversations to learn how to address issues, how to talk to customers, how to properly escalate, how to behave in the interaction realm, and all in the business language of your own brand and company! This would be impossible for any human being, but for a bot…well, no big deal.

And the result: a perfectly trained agent ready to respond to your most difficult inquiries like your best skilled agent. Also, because every contact centre is different from the other, their recordings will result in different and more accurate learning and behaviour of the same AI. Isn’t AI in such case a dream come true?

As consumers, we probably would not care that the responses come from a bot, especially with digital channels where there is no voice and tracking a bot might be really tricky, and in the end, what matters most is the CX perception, not the reality. 🙂