What challenges do UMI developers face?“If a user has to click more than one button to start working with a product, they won’t want to do it”Dear friends, we have always tried and continue to try to create products that will be the most high-tech, easy-to-understand, secure and user-friendly for any user. In today’s article, we want to share with you the problems that we, UMI developers, face, as well as solutions we find. After all, you only see the final product — an app that is convenient to interact with on a smartphone or computer. However, “under the hood”, this app has a complex infrastructure with a range of solutions.
Today, we will tell you about the path the ready-to-use program code has to go before our users can start using it. It seems obvious that a nice and operating code is not enough to create a convenient product. We need to assemble and package the program in such a way that it is convenient for the user to work with.
A variety of operating systems and architectural platformsThere are a huge number of operating systems based on various architectural platforms, but it does not make any sense to try and create solutions for all of them at once. It would be more logical to target the most popular systems when creating an app.
Historically, three operating systems have become the most popular: Windows, Linux and macOS. However, they can be run on different hardware which is why a new variable appears — the processor architecture.
Previously, the mostly used processors included i386 (IA-32, 32-bit) and AMD64 (x86-64, 64-bit). However, as various mobile devices became more popular, active development of the ARM architecture platforms began, which also took the path of improvement from 32-bit to 64-bit version.
For the user, there’s just one difference between the 32-bit and 64-bit versions — the latter allows for the use of more than 4 GB of RAM with no particular difficulties. Since we knew that the UMI network could process a huge amount of transactions, we realized that network nodes would consume more memory. Accordingly, running a node with a small amount of addressable memory makes no sense in the long term.
Based on the above, we decided to support 64-bit architectures only. In practice, this means supporting four operating systems and corresponding processor architectures:
- Windows OS and AMD64 processor architecture;
- Linux OS and ARM64 processor architecture;
- Linux OS and AMD64;
- Two options for Apple devices that can be combined into one: Darwin operating system and ARM64 processor architecture (for M1 processors) or AMD64 processor architecture (for Intel processors).
We have managed to get rid of external dependencies which streamlines the assembly of the software core — the Go (golang) compiler is all we need.
However, the difficulties are still there because we need to pack the app into a package. Let’s consider a solution to this problem using Linux OS as an example.
There are a huge number of Linux distribution kits. They mainly differ by the set of libraries included in the distribution kit. This is where difficulties arise. For example, if you build an app with dependencies for CentOS 5, it may not work on a more recent release due to the different version of the libraries the app depends on.
Therefore, we are building the UMI app with no dynamic dependencies, that is, it does not depend on the environment it is launched in, at all.
It may seem it has solved our problem — we just need to download or compile a binary UMI file from the source files and run it. But it’s actually a little more complicated than that. In fact, our entire architecture was initially built around the Kubernetes open source software. Therefore, we used Docker as a packaging tool. We assembled Docker images for the AMD64 platform, and it all worked just fine. Docker was also good in a local environment.
However, a different Docker version for the new Apple M1 platform has recently been released, so we had to assemble two versions of the package — for AMD64 and ARM64. At first, it went smoothly thanks to buildx, an experimental Docker functionality. But following transition to the binary blockchain, we had to abandon Kubernetes because we had new requirements for the speed of the disk subsystem. Since we use Amazon Linux 2 based on the RHEL distribution kit to run virtual machines in the Amazon cloud, we decided to try using Advanced Packaging Tool (APT), a regular package manager, and RPM.
Initially, we had no problems with assembling RPM packages using Docker for the AMD64 platform. Later, while optimizing the server architecture cost, we tried using AWS instances on the ARM64 platform. To do this, we had to assemble RPM packages for the ARM64 architecture. Buildx handled the task perfectly well.
At around the same time, the UMI community asked us to add support for the Yum package manager used in Debian and Ubuntu. We delivered. As a result, we now have our own package repository —
https://pkg.umi.top/.
So, while preparing each release, we assembled three versions of Linux packages — each package designed for two architectures. In total, it means six stages of development, if we leave out the pipelines for updates and repositories synchronization.
Later we ran into some issues:
- After each update of macOS (used by our developers as the main OS), the programs we used worked worse and worse.
- After another Android Studio update, the network stopped working in the Android emulator;
- We had some problems with network operation in VirtualBox which we actively used to test network interaction.
Working in such conditions became more and more difficult — we had to spend more and more time on repairing the tools rather than on the work itself. When we say ‘tools’, we mean compilers (packagers) we use to prepare products for release.
We decided to try the option with a test environment based on Google Cloud and created a new account to do this. In practice, however, it turned out that this has no major advantages, so we abandoned this idea.
Step by step, we came to the option with an ESXi hypervisor running on a local computer. We liked this option the most, so we just settled on it. But then, we still had the problem of assembling packages for different architectures. The most logical option would be to run separate servers to assemble packages for the target architecture. This would mean maximum efficiency with a relatively small increase in infrastructure costs.
But the recent instability in the world has forced us to reconsider the idea of using “foreign” servers for critical services. Therefore, we started looking for a solution that would allow us to solve the problems with assembling all the necessary packages and updating repositories within one local server. This solution would have to be easily reproducible in the event we need to move all the components to another environment.
As a result, GoReleaser became the solution of our choice. The first impression was quite good. We are now gradually transferring assembly lines to this tool. As we work, we carefully review and test everything; we also have to tackle with multiple peculiar things. It takes a lot of time, but so far this approach seems the most rewarding.
Dear friends, as you can see, the development of complex blockchain products is a time-consuming process most users don’t even notice. We are constantly facing new challenges and sometimes we spend a lot of resources and creative energy on solving the emerging problems. However, it’s definitely worth it as we want to create the best-of-class product that would be accessible to as many users as possible.
In the future, we will continue to be in close touch with you and publish technical articles to share our experience. We hope it’s been interesting.
Love, UMI Team!
