Skip to main content

· 5 min read

本文的中文版发布在微信公众号,点击 《2022,云上开发的新纪元》即可阅读,期待您的订阅~

Remote Dev Buzz on Social Media

A recent tweet from Kelsey Hightower once again made a lot of buzz on social media. His tweet underlined that the limitation of local resources and the complexity of external dependencies have made remote development trending.

Kelsey Hightower

Short as the tweet is, it has great repercussions for the developer community. Hundreds of people followed up on Twitter complaining about local development.

cq

Coincidentally, @swyx from Temporal also published an article recently about "The Death of Localhost".

the death

Why do all the tech leaders start to promote remote development? In this article we will analyze the history of such technology and what the future looks like.

More Developer Tools Moving to Cloud

moving

Software development lifecycle (SDLC) is conventionally divided into two processes: the inner loop and the outer loop. The inner loop includes coding, testing, building until git push. The outer loop covers the rest of the steps from git push to a release of the software.

In the past, it was mostly local tools that the developers used in the inner loop phase, because network latency could reduce the development efficiency and developers want fast feedback.

However, this boundary is gradually breaking down with more developer tools being cloud-enabled. That is because developers spend most of their time not on writing code, but finding the correct solutions. If a cloud service can help them find a better solution more quickly, then 100ms latency is anything but a problem. Today there are a couple of cloud-enabled examples:

  • Github Copilot autocompletes your code using AI powered analysis and predictions. It is so intelligent that many developers said they searched StackOverflow less after using it.
  • Sourcegraph provides cloud-enabled code search. Developers say that they do a better job of searching for theircode with Sourcegraph than what they do locally.
  • Cloud Shell is an online terminal that contains the common utilities for its cloud provider (e.g. awscli). It makes it easier for users to learn and experience the cloud.

Brand new experience on Cloud

In the inner loop, the hardest part for users is setting up the dev environment. Because it is time consuming, if not too hard to achieve, to install complex microservice dependencies, and configure numerous infrastructure components. What's more, for developers working on open source or agile schedule, it is their first time engaging in a project, which means setting up a dev environment is nothing but a trouble. Last but not least, even if developers have successfully setup the environment in local, there is still a very large gap between the local and production environment -- that means more troubles and failures in production releases.

To boost dev efficiency, more and more companies are moving dev environments to the cloud:

  • Bigcos : Big companies like Google、FB、Etsy、Tesla、Shopify spin up a development environment according to the needs of developers so that the whole process (coding, testing, building, git pushing, releasing of the software) could be done on cloud.
  • Small and medium-sized enterprises (SMEs):More and more SMES are buying services offered by companies like Github Codespace、Gitpod、StackBlitz、Okteto to manage cloud dev environments.
  • In-house solutions based on open source projects: Projects like Nocalhost provide a cloud-native remote development experience based on containers. It has helped a number of enterprises build in-house management platforms for dev environments.

ymqd

Providing developers the seamless workflow for onboarding application development, without the painful process of setting up dev environments, is a proven track to boost dev efficiency. It is becoming a trend adopted by more and more companies.

Today, an ideal cloud-native dev experience should be like this:

  • One-click deploy:The entire environment stack is predefined, and developers can self-service to spin up the environment quickly. In this way, users can enjoy developing apps after a few clicks of choosing programming frameworks and backend services like MySQL, Redis, Prometheus.
  • Cattle, not pet: Dev environments should be cattle, not pet. Every single environment can be codified, reproducible, and immutable.
  • On-dema: The workflow is designed on a developer-centric basis, allowing developers to spin up environments on demand by forking a branch. Once the development is finished, branch is merged to main, the environment will be deleted automatically.
  • Integrated experience: Build higher level IDE based on VSCode, Jetbrains which fit the developer use cases. For example, making it easy to share the environment, create an issue by clicking IDE, and split traffic using GUI, and so on.

flow

At Heighliner we envision this and want to make it a reality. We build projects and cloud services to provide a seamless cloud dev experience.

If you are interested in Heighliner, Dagger, Nocalhost, or any other cloud native projects, you are welcome to:

· 6 min read

alt

Today the cloud takes us to a new era: With an increasing number of cloud infrastructure services, a dedicated team of developers can ship a compelling product in almost any industry within weeks. The cloud has become something like water and electricity for the entire human society. We have witnessed the rapid growth of the cloud especially when COVID hit the world.

Along with it comes the fast-growing cloud native community. Cloud native technologies are changing the way developers use cloud services via open standards. In this way, developers can build their apps across clouds without locking into specific vendors. Since Kubernetes donation as the first project, CNCF (Cloud Native Computing Foundation) has been cradling and growing hundreds of projects as a neutral foundation. It is still evolving fast and providing developers with more standard interfaces to use the cloud. By now, cloud native means not only adopting Kubernetes, but also building best practices for the entire software development lifecycle. Here we are listing some of the latest trends:

  • DevSecOps: It continues to be the primary focus of IT management and software engineering teams to secure the cloud infrastructure. As software supply chain grows and privacy policies improves, there will be security and conformance validation in every step of software delivery and maintenance.
  • Remote Development: Developers are often limited by the hardware resources of local laptops, inconsistency between development and production environments, and complication of microservice dependencies. Nonetheless, projects like Nocalhost are trying to move development activities onto the cloud. All developers need is an IDE and they can even collaborate on the same environment.
  • Observability: As OpenTelemetry becomes more and more popular, the format to collect logs and metrics becomes more standardized. This has improved the development of digital transparency of cloud apps. Meanwhile, uprising continuous profiling projects are making it easier to do problem debugging and performance analysis. It takes them to a new level that you can see what happened simultaneously. Moreover, the rise of ePBF adoption in observability makes it possible to observe with no instrumentation, which frees developers and improve their efficiency.
  • GitOps: CI/CD gradually becomes the primary topic as the concepts of cloud native gets widespread. GitOps, as a new pattern for continuous delivery from the cloud native community, is trying to codify into Git all the infrastructure dependencies of your app in a declarative format. By doing so developers will get better version control and access management for their application infrastructure. Recently, the founders of ArgoCD project has created a startup (akuity.io) for commercializing ArgoCD. This reflects the growing demand for GitOps and confidence from the capital markets.

Are Developers Satisfied?

Even though having so many choices from the cloud native community, developers still have complex feelings:

  • On the one hand, they like these choices and to use them in production. In this way they can build best practices based on open standards, and have a future-proof application architecture.
  • On the other hand, they are scared and overwhelmed by the complexities of open source projects. It is very complicated and difficult to setup these projects from end to end. Moreover, the iteration speed of these projects is very high. Developers don't have enough time and energy to keep up with.

Heighliner Born For Developers

Developers' time are precious. If they can't focus themselves on developing applications but waste too much time on getting infrastructure tools to work, it will eventually incur huge cost to both developers and enterprises.

Today we build a developer tool called Heighliner to solve the above problem. It is a modern developer tool that can deliver your application stack as code. You can codify low level details into human-readable configuration files that you can version, reuse, and share. We try to tell developers via Heighliner that "Your application stack can be managed just like your application code. You deserve a better user experience." We say that because Heighliner is:

  • Developer-centric: Heighliner defines the entire application stack from a developer-centric perspective, including code repositories, app frameworks, CI/CD pipelines, Helm Charts, monitoring dashboards, alerting rules, remote development config, and so on.
  • Versatile: We provide official stacks for popular languages and frameworks. Users can choose one of these stacks to spin up cloud native development environments in one click. The languages and frameworks include Go, Spring, Next.js, Vue.js, Remix, etc.
  • Customizable: Users can customize every single step of any stack. We are glad that Docker creator Solomon has created Dagger. Dagger significantly simplify our user experience to develop Heighliner stacks. We support using Dagger to customize any stack. Besides Dagger libraries, we provide additional libraries targeted for cloud native technologies.

With above benefits from Heighliner, developers can take advantage of the evolution of cloud native technologies. The latest technological updates will be consumed in a way that developers are familiar with -- using code. In this way developers can continuously evolve their application architecture.

Getting Hands-on

In the following we will demonstrate how to spin up an entire cloud native development environment within 2 minutes.

Step 1: Install Heighliner tool

$ curl -L https://dl.h8r.io/hln/install.sh | sh

Step 2: Prepare a Kubernetes cluster

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-24T22:10:16Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/arm64"}

Step 3: Create your development environment in one command

$ hln up hello-world -s sample -i

alt

Eventually Heighliner will build and deploy your application:

alt

Heighliner will create automatically for you the Github repositories, Helm Chart, CI/CD pipelines, DNS domain routing, ArgoCD apps, Grafana dashboards, Nocalhost DevSpace, etc.

alt alt alt

Embrace Community

By now, Heighliner has provided many out-of-box features that you can play with. You can learn more at heighliner.dev.

In the mean time, we believe in the long term value that open source community will bring us. All projects of Heighliner including official stacks are open source Apache-licensed. Moreover, we are contributing to open source projects like Dagger, Nocalhost, etc. It is our ambition to build an open community that benefits every developer in the future.

Heighliner is still in the early stages of development. To grow and prosper it needs badly one important factor -- you as the passionate developer. If you are interested in Heighliner, Dagger, Nocalhost, or any other cloud native projects, you are welcome to:

· 4 min read
Zhenwei Wang

The Heighliner Stack provides an easy way to build complex cloud-native applications. It makes sure cloud native applications are developed and deployed with best practices.

simplicity

Complexity of building cloud-native applications

The complexity of application architecture multiplies as the business grows. Say you start building a simple personal blog hosted on the cloud, what platform and tech stack would you choose? The answer might be Heroku, WordPress, Netlify, or even GitHub pages. They are simple, cheap, and reliable enough for simple applications such as personal blogs. In this case, no one cares about micro-service, containers, Kubernetes, Autoscaling, GitOps, so on and so forth.

archtecture

Nonetheless, modern enterprises, such as e-commerce and/or social platforms, have employed thousands of developers to build software in a monolithic application. As the number of developers grows, so does the complexity of building such an application. The micro-service architecture supports separating concerns of different teams and better cooperation. As a result, micro-service architecture gets more and more popular.

To support such micro-service trends, cloud native technologies has been born to satisfy modern application architecture. Developers want their applications to be highly available and fault-tolerant. But the number of applications gets 100x bigger so they can't be managed and operated in the old way. Therefore, concepts such as elastic scaling, distributed monitoring, and deployment strategies have emerged to solve such problems.

cloud-native

Developers also want their applications to be highly available and fault-tolerant. But micro-service pattern will increase the number of applications to 100x bigger. This will make it hard if not impossible to manage or operate. Concepts such as service mesh, distributed tracing, continuous profling has emerged. Developers has been given the power to easily build CI/CD pipelines, elastic scaling solutions, observability stacks, canary deployment strategies, etc.

With great power comes great responsibility. Now the developers realize they spend more time dealing with archetecture than buisness code. Soon they realize this is wrong -- they need to focus on buisness logic that generates revenue! This is a new problem that developers face in the cloud native journey.

Configuration! Configuration! Configuration!

The sheer number of tooling in cloud native community is growing fast. Soon developers are overwhelmed by the configuration work.

Here is the Cloud Native Landsacpe of CNCF (https://l.cncf.io/)

simplicity

Turning this picture into a production toolchain requires tons of configuration. Terraform provides a good way to ochestrate the infrastructure. Kubernetes provides a good way to ochestrate the deployment. But there is still no good way to ochestrate the archetecture and DevOps workflow.

Consider the following questions:

  • Should the source code and the Kubernetes Manifests be in the same Git repository?
  • What is the mapping relationship between branches and environments?
  • What namespace monitoring metrics should Prometheus capture?
  • What dashboard and graph should Golang developers focus on?
  • What alert rules should be configured?
  • Should I use a self-built open source product or a commercial product?
  • How to remote debug container in Kubernetes cluster?

cloud-stack

Everyone wants to know the industry standards to make their decisions. Thus we need well-defined cloud-native best practices as a collection of all proven and reliable methods.

Preconfigured Best Practices

We build these best practices into a project called Heighliner. Starting from developer-centric mind, we provides many pre-configured stacks covering popular programming languages and frameworks. For example, gin-vue is a toolchain preconfigured with the following tools and techniques:

  • Golang
  • Git Repository(Currently provided by GitHub)
  • Gin Framework
  • Swagger
  • Vue.js
  • Gorm
  • Docker
  • CI pipeline(Currently provided by GitHub Actions)
  • ArgoCD
  • Mariadb
  • Redis
  • Helm
  • Kubernetes
  • Nocalhost
  • Prometheus
  • Grafana
  • Jaeger
  • Loki

A toolchain composed of the above tools will support business development based on Golang. Preconfigured delivery flow speeds the DevOps process up. Pre-configured observation capabilities covering logs, metrics and distributed tracing. Database and messaging middleware ensures applications can handle stateful and asynchronous tasks with ease. GitOps based on ArgoCD and helm makes application publishing simple and efficient.

gin-vue-stack

Based on a stack like this, developers can spin a cloud-navite application up in minutes without the complexcity of configuration. Stack is open source and transparent, which ensures that developers have no barriers to understand all the process of code from editing to deployment.

With this convention-over-configuration mindset, developers can enjoy the coding-testing-commiting inner development loop.

What's next?

A developer console can be built to display key states in delivery flow, such as build results, deployment versions, logs, traffic control rules, alerts etc. Developers can get any useful information in that console and can manipulate the application the way they want. Also, developers can customize by forking a copy on GitHub to meet their special needs.