2 posts tagged with "Stack"

本文的中文版发布在微信公众号，点击 《2022，云上开发的新纪元》即可阅读，期待您的订阅～

Remote Dev Buzz on Social Media​

A recent tweet from Kelsey Hightower once again made a lot of buzz on social media. His tweet underlined that the limitation of local resources and the complexity of external dependencies have made remote development trending.

Short as the tweet is, it has great repercussions for the developer community. Hundreds of people followed up on Twitter complaining about local development.

Coincidentally, @swyx from Temporal also published an article recently about "The Death of Localhost".

Why do all the tech leaders start to promote remote development? In this article we will analyze the history of such technology and what the future looks like.

More Developer Tools Moving to Cloud​

Software development lifecycle (SDLC) is conventionally divided into two processes: the inner loop and the outer loop. The inner loop includes coding, testing, building until git push. The outer loop covers the rest of the steps from git push to a release of the software.

In the past, it was mostly local tools that the developers used in the inner loop phase, because network latency could reduce the development efficiency and developers want fast feedback.

However, this boundary is gradually breaking down with more developer tools being cloud-enabled. That is because developers spend most of their time not on writing code, but finding the correct solutions. If a cloud service can help them find a better solution more quickly, then 100ms latency is anything but a problem. Today there are a couple of cloud-enabled examples:

• Github Copilot autocompletes your code using AI powered analysis and predictions. It is so intelligent that many developers said they searched StackOverflow less after using it.
• Sourcegraph provides cloud-enabled code search. Developers say that they do a better job of searching for theircode with Sourcegraph than what they do locally.
• Cloud Shell is an online terminal that contains the common utilities for its cloud provider (e.g. awscli). It makes it easier for users to learn and experience the cloud.

Brand new experience on Cloud​

In the inner loop, the hardest part for users is setting up the dev environment. Because it is time consuming, if not too hard to achieve, to install complex microservice dependencies, and configure numerous infrastructure components. What's more, for developers working on open source or agile schedule, it is their first time engaging in a project, which means setting up a dev environment is nothing but a trouble. Last but not least, even if developers have successfully setup the environment in local, there is still a very large gap between the local and production environment -- that means more troubles and failures in production releases.

To boost dev efficiency, more and more companies are moving dev environments to the cloud:

• Bigcos : Big companies like Google、FB、Etsy、Tesla、Shopify spin up a development environment according to the needs of developers so that the whole process (coding, testing, building, git pushing, releasing of the software) could be done on cloud.
• Small and medium-sized enterprises (SMEs)：More and more SMES are buying services offered by companies like Github Codespace、Gitpod、StackBlitz、Okteto to manage cloud dev environments.
• In-house solutions based on open source projects: Projects like Nocalhost provide a cloud-native remote development experience based on containers. It has helped a number of enterprises build in-house management platforms for dev environments.

Providing developers the seamless workflow for onboarding application development, without the painful process of setting up dev environments, is a proven track to boost dev efficiency. It is becoming a trend adopted by more and more companies.

Today, an ideal cloud-native dev experience should be like this:

• One-click deploy：The entire environment stack is predefined, and developers can self-service to spin up the environment quickly. In this way, users can enjoy developing apps after a few clicks of choosing programming frameworks and backend services like MySQL, Redis, Prometheus.
• Cattle, not pet: Dev environments should be cattle, not pet. Every single environment can be codified, reproducible, and immutable.
• On-dema: The workflow is designed on a developer-centric basis, allowing developers to spin up environments on demand by forking a branch. Once the development is finished, branch is merged to main, the environment will be deleted automatically.
• Integrated experience: Build higher level IDE based on VSCode, Jetbrains which fit the developer use cases. For example, making it easy to share the environment, create an issue by clicking IDE, and split traffic using GUI, and so on.

At Heighliner we envision this and want to make it a reality. We build projects and cloud services to provide a seamless cloud dev experience.

If you are interested in Heighliner, Dagger, Nocalhost, or any other cloud native projects, you are welcome to:

• join our Discord channel
• contact us via community channels.
• provide us feedback at heighliner github issues

The Heighliner Stack provides an easy way to build complex cloud-native applications. It makes sure cloud native applications are developed and deployed with best practices.

Complexity of building cloud-native applications​

The complexity of application architecture multiplies as the business grows. Say you start building a simple personal blog hosted on the cloud, what platform and tech stack would you choose? The answer might be Heroku, WordPress, Netlify, or even GitHub pages. They are simple, cheap, and reliable enough for simple applications such as personal blogs. In this case, no one cares about micro-service, containers, Kubernetes, Autoscaling, GitOps, so on and so forth.

Nonetheless, modern enterprises, such as e-commerce and/or social platforms, have employed thousands of developers to build software in a monolithic application. As the number of developers grows, so does the complexity of building such an application. The micro-service architecture supports separating concerns of different teams and better cooperation. As a result, micro-service architecture gets more and more popular.

To support such micro-service trends, cloud native technologies has been born to satisfy modern application architecture. Developers want their applications to be highly available and fault-tolerant. But the number of applications gets 100x bigger so they can't be managed and operated in the old way. Therefore, concepts such as elastic scaling, distributed monitoring, and deployment strategies have emerged to solve such problems.

Developers also want their applications to be highly available and fault-tolerant. But micro-service pattern will increase the number of applications to 100x bigger. This will make it hard if not impossible to manage or operate. Concepts such as service mesh, distributed tracing, continuous profling has emerged. Developers has been given the power to easily build CI/CD pipelines, elastic scaling solutions, observability stacks, canary deployment strategies, etc.

With great power comes great responsibility. Now the developers realize they spend more time dealing with archetecture than buisness code. Soon they realize this is wrong -- they need to focus on buisness logic that generates revenue! This is a new problem that developers face in the cloud native journey.

Configuration! Configuration! Configuration!​

The sheer number of tooling in cloud native community is growing fast. Soon developers are overwhelmed by the configuration work.

Here is the Cloud Native Landsacpe of CNCF (https://l.cncf.io/)

Turning this picture into a production toolchain requires tons of configuration. Terraform provides a good way to ochestrate the infrastructure. Kubernetes provides a good way to ochestrate the deployment. But there is still no good way to ochestrate the archetecture and DevOps workflow.

Consider the following questions:

• Should the source code and the Kubernetes Manifests be in the same Git repository?
• What is the mapping relationship between branches and environments?
• What namespace monitoring metrics should Prometheus capture?
• What dashboard and graph should Golang developers focus on?
• What alert rules should be configured?
• Should I use a self-built open source product or a commercial product?
• How to remote debug container in Kubernetes cluster?

Everyone wants to know the industry standards to make their decisions. Thus we need well-defined cloud-native best practices as a collection of all proven and reliable methods.

Preconfigured Best Practices​

We build these best practices into a project called Heighliner. Starting from developer-centric mind, we provides many pre-configured stacks covering popular programming languages and frameworks. For example, gin-vue is a toolchain preconfigured with the following tools and techniques:

• Golang
• Git Repository(Currently provided by GitHub)
• Gin Framework
• Swagger
• Vue.js
• Gorm
• Docker
• CI pipeline(Currently provided by GitHub Actions)
• ArgoCD
• Mariadb
• Redis
• Helm
• Kubernetes
• Nocalhost
• Prometheus
• Grafana
• Jaeger
• Loki

A toolchain composed of the above tools will support business development based on Golang. Preconfigured delivery flow speeds the DevOps process up. Pre-configured observation capabilities covering logs, metrics and distributed tracing. Database and messaging middleware ensures applications can handle stateful and asynchronous tasks with ease. GitOps based on ArgoCD and helm makes application publishing simple and efficient.

Based on a stack like this, developers can spin a cloud-navite application up in minutes without the complexcity of configuration. Stack is open source and transparent, which ensures that developers have no barriers to understand all the process of code from editing to deployment.

With this convention-over-configuration mindset, developers can enjoy the coding-testing-commiting inner development loop.

What's next?​

A developer console can be built to display key states in delivery flow, such as build results, deployment versions, logs, traffic control rules, alerts etc. Developers can get any useful information in that console and can manipulate the application the way they want. Also, developers can customize by forking a copy on GitHub to meet their special needs.