Cloud Engineering – Software Engineering Daily

On January 31st 2017, GitLab experienced a major outage of their online repository hosting service. The primary database server experienced data loss due to a combination of malicious spam attacks and engineering mistakes that occurred while trying to respond to those spam attacks. GitLab responded to the event transparently. The company put up a postmortem The post GitLab with Pablo Carranza appeared first on Software Engineering Daily.

Healthcare is a complex business. Oscar is a company that wanted to build a new insurance provider–but realized that healthcare is so interconnected that in order to build a new insurance provider, realized it actually needed to build an entire healthcare business too, complete with patient management and facilities. Since Oscar is a modern technology The post Healthcare Engineering with Isaac Councill appeared first on Software Engineering Daily.

Many companies are transitioning from a monolith to microservices architecture. Tools for cloud computing, containerization, and continuous delivery are making this easier. But there are still technological and organizational challenges that a company will encounter while making this transition. Cassandra Shum is an engineer with ThoughtWorks. She has worked with major financial institutions and other The post Microservices Transition with Cassandra Shum appeared first on Software Engineering Daily.

Firebase is a backend-as-a-service. The key efficiency of a backend-as-a-service is that it enables developers to go from having a 3-tier architecture (client, server, database) to a 2-tier architecture (client, backend-as-a-service). The team who started Firebase built it as a pivot. They had started a social network, and then they realized there wasn’t a good The post Firebase with Doug Stevenson appeared first on Software Engineering Daily.

Spring Framework is an application framework for Java and JVM languages. Spring was originally built around dependency injection, but grew to become an entire ecosystem of tools and plugins for Java developers. Spring was originally released 15 years ago, and since then a lot has changed around application development. For example, many engineers deploy applications The post Spring Boot with Josh Long appeared first on Software Engineering Daily.

The word “microservices” started getting used after a series of events–companies were moving to cloud virtual machines. Those VMs got broken up into containers, and the containers can fit to the size of the service. Services that are more narrowly defined take up smaller containers, and can be packed more densely into the virtual machines–hence The post Microservices Practitioners with Austin Gunter and Richard Li appeared first on Software Engineering Daily.

Search is a common building block for applications. Whether we are searching Wikipedia or our log files, the behavior is similar: a query is entered and the most relevant documents are returned. The core data structure for search is an inverted index. Elasticsearch is a scalable, resilient search tool that shards and replicates a search The post Elasticsearch with Philipp Krenn appeared first on Software Engineering Daily.

There are various standards at play when creating and consuming Application Program Interfaces (APIs).  These standards, though, are mostly technical and mostly lower-level than the content of the API. Andy Beier has experienced the broad range of API quality in his role with Domo in creating integrations with other businesses.  He has made standardization of The post API Design Standards with Andy Beier appeared first on Software Engineering Daily.

Servers in a data center fail. Sometimes entire data centers have a power outage. Bugs in an application make it into production. Human operators make mistakes and cause data to be deleted. Failure is unavoidable. We make backups and replicate our servers so that when a failure occurs, we can quickly respond to it without The post Failure Injection with Kolton Andrus appeared first on Software Engineering Daily.

Designers and software engineers need to communicate with each other. From Apple to Slack to Uber, the emphasis on visual design within a product is rising in importance. Much like development and operations siloes have been bridged with the DevOps movement, design and engineering teams are working more closely together to align the vision of The post Software Psychology with Bjorn Freeman Benson appeared first on Software Engineering Daily.

If you are building a service that processes payments, your software architecture has a lot of requirements. Not only do you need to be highly available, consistent, and fast–you need to be PCI compliant. In this episode, we explore the infrastructure of Stripe with Evan Broder, who has been with the company for five years. The post Stripe Infrastructure with Evan Broder appeared first on Software Engineering Daily.

Observability allows engineers to understand what is going on inside their systems. In its most raw form, observability comes from log data. Modern systems have many layers of logs–virtualized cloud infrastructure, container orchestration, the container runtime itself, and the application logic running within the container. With all of these layers, it is not practical for The post Stripe Observability with Cory Watson appeared first on Software Engineering Daily.

Command Query Responsibility Segregation (CQRS) is a powerful concept that has the potential to make for reliable and maintainable systems.  It is also broadly misunderstood and means different things to different people. Derek Comartin learned about the idea after viewing some talks by Greg Young and has since successfully applied the approach with great success The post Using CQRS to Make Controllers Lean with Derek Comartin appeared first on Software Engineering Daily.

Load testing measures performance of a system undergoing a large volume of requests. Before an application is pushed to production, engineers will often load test their software to ensure it is resilient in the face of high traffic. As web applications have changed, the requirements around load testing have changed as well. External APIs, internal The post Load Testing with Mark Gilbert appeared first on Software Engineering Daily.

Parse was a backend as a service company built in 2011 before being acquired by Facebook in 2013. Building a backend as a service for developers requires walking a thin line between giving engineers lots of control and preventing those engineers from shooting themselves in the foot. While she was at Parse, Charity Majors learned The post Parse and Operations with Charity Majors appeared first on Software Engineering Daily.

When an application is using all of its available resources, that application needs to be scaled. Scaling an application means giving it more resources–typically servers. Autoscaling is an engineering practice where an application is automatically given more or less resources based on how healthy the application performance is at a given time. Applications on Heroku The post Heroku Autoscaling with Andrew Gwozdziewycz appeared first on Software Engineering Daily.

In the mid 90s, data warehousing might have meant “using an Oracle database.” Today, it means a wide variety of things. You could be stitching together a big data pipeline using Kafka, Hadoop, and Spark. You could be using managed tools like BigQuery from Google. How did we get from the simple days of Oracle The post Data Warehousing with Mark Rittman appeared first on Software Engineering Daily.

Most tech companies are moving toward a highly distributed microservices architecture. In this architecture, services are decoupled from each other and communicate with a common service language, often JSON over HTTP. This provides some standardization, but these companies are finding that more standardization would come in handy. At the ridesharing company Lyft, every internal service The post Service Proxying with Matt Klein appeared first on Software Engineering Daily.

Infrastructure is a term that can mean many different things: your physical computer, the data center of your Amazon EC2 cluster, the virtualization layer, the container layer–on and on. In today’s episode, podcasters Chris Wahl and Ethan Banks discuss the past, present, and future of infrastructure with me. Ethan and Chris host Datanauts, a podcast The post Infrastructure with Datanauts’ Chris Wahl and Ethan Banks appeared first on Software Engineering Daily.

Giphy is a search engine for gifs, the short animated graphics that we see around the Internet. Giphy is also a creative platform where people create new gifs. Every search engine requires the construction of a search index, which is a data structure that responds to search queries efficiently. Since Giphy is a search engine The post Giphy Engineering with Anthony Johnson appeared first on Software Engineering Daily.