Richie’s Techbits newsletter: Issue 4: What language does farm equipment speak? Should have used Pocketbase and the 3 types of OSS

How does farm equipment communicate?

Would it surprise you to learn that agricultural equipment and, from what I can gather, most manufacturing equipment, uses a protocol that has remained largely unchanged from 1980s?

Depending on your background it may surprise you that farm equipment contains networked computer equipment at all. However it does and I’m not talking about weather stations that transmit readings over LoRaWAN or things that can be secreted on top of existing infrastructure.

I’m talking about heavy duty industrial equipment such as wood chip boilers and heat distribution systems for green houses amongst a host of other things. These devices often have ethernet connections and sometimes VNC for remote access to the display panel. VNC is not the best protocol for collecting telemetry and coordinating the operation of multiple devices however. For this these machines provide Modbus over TCP or RS242/RS485 (i.e. a serial cable).

My experience is with a single farm, but practically all of the equipment supports Modbus with one or two exceptions. This is exciting because it means there is a way to create a centralised system that collects telemetry data and tune equipment based on feedback. The problem is that Modbus is a very basic protocol that just allows setting and reading registers, what the data in each register means is machine dependent and either requires a register map or reverse engineering to understand.

A second issue is finding a way to plug in to Modbus when it is already connected to the display panel or a PCL via serial. This may require physically interfering with the machine to install a Modbus to ethernet gateway or indeed a Modbus to MQTT gateway or something else with a modicum of security. On the plus side there are a lot of modules which can be bought off the shelf to facilitate this.

Now for those of you who are used to working with a Kubernetes cluster with full observability tooling or work in data engineering. Imagine going from having a bunch of disparate systems that require a human to run between them to communicate, to a centralised observability and data analysis platform with all the tools developed for the cloud and big data?

At this point I’m not talking about doing anything really fancy, just having a Grafana or Metabase dashboard with everything that happened that day to all of the devices and sensors would be a huge improvement. In high end motor sport it’s required to have telemetry from every part of the vehicle just to be competitive and I don’t believe having fast and detailed telemetry would be any less valuable in agriculture.

While all of the machines are digitised and even networked, they are still essentially islands and if we extrapolate this to many other industries it’s easy to see the potential. There are many networked machines out there which only require minor hardware updates and some software to be entered into an integrated whole. The major challenge is that every machine is different, Modbus isn’t self describing and you can be sure most of the equipment manufacturers don’t publish the docs in a convenient location.

As Karl Marx observed, technology or human progress in general, advances unequally. In some areas we are racing ahead with LLMs, that are going to replace you next year by the way, and in others we are trying to get a machine to report sensor readings using technologies that have barely changed since the 1970s.

Should have used Pocketbase

One of my regrets when creating dobu.uk was that I didn’t use something like Pocketbase. Instead I recreated a whole bunch of plumbing needed by any web app. What you get with Pocketbase:

• Auth and user management, meaning you don’t have to implement passwords or OAuth2 endpoints
• Collections/Records API that abstracts away SQLite, meaning you don’t have to write SQL (unless you want to) nor deal with an ORM
• Optional automatic database migrations, meaning you don’t have to write migration scripts or schedule them
• Inbuilt admin and DB UI, meaning you can create your schema, query and update data from in a GUI without reaching for any other tools
  
• Scale to 1000’s of concurrent connections on a low cost VM
• Use it as a HTTP API server, as a Go web framework or both, meaning the API can be extended with Go
• Completely self contained and easy to host

There is more stuff like Cron jobs, Backups and so on. On the downside it is pre-1.0 and they specifically warn of breaking changes. Breaking changes are always a risk in web development, but if they actually warn you of it then I tend to err on the side of caution. At least now I err on the side of caution after being burned by SvelteKit.

I assume that Pocketbase is a play on the names of Firebase and Supabase. All of these abstract away a lot of the plumbing associated with making even the most trivial of SaaS. Unless an app is completely stateless then people who are used to, ahem, writing zero-dependency-low-level code where 1ms is a long time can quickly get into a mess when they see the disgusting mess that JavaScript libraries represent and thus decide to do everything themselves.

The reality is one may be able to produce something better in one particular area. Let’s say an area of very great importance to one’s particular application. However everything else should be accepted as it is because there just isn’t enough time to sort through all of the details and craft a solution specific to your needs. This is a very domain dependent statement and that was the issue for me, because when crafting some fundamental low level code it can absolutely make sense to pay attention to every detail.

Pocketbase makes no attempt to scale horizontally and this is another important point. I wanted to have my apps data replicated around the world so that it could be accessed with low latency in any location. I got hooked on the idea of doing this at the database level and using eventual consistency for most operations. I managed to do it using KeyDB, but its only benefit was an educational one.

I have to admit that basic caching could have solved this problem in my particular app, even just a CDN like KeyCDN could do most of it I reckon. If the VM running Pocketbase goes down this could be an issue, but if most functions of the website keep running from the cache, it’s not a huge one. Finally, if you really need to scale horizontally it can be done at the application level.

Finally Pocketbase is a True Open Source project rather than a sales funnel for something else. This got me thinking about Open Source again and the different types of Open Source which I’ll explain below.

The three types of Open Source

I woke up in the middle of the night, heart pounding and sweating and thought “There are three types of Open Source”. I think I had eaten too much chocolate before going to bed and was having an insulin crash, but obviously this must have been percolating somewhere in the back of my mind.

1. The Sales Funnel
2. Art
3. The Collaboration

The Sales Funnel is an Open Source project where the motivation to maintain and develop it comes from the marketing value of being the developer of the Open Source software. For the obvious reason that Open Source can not be sold, there is no way to directly make money from it, however one way you can extract value from it is by integrating it into your sales funnel.

If the Art project encompasses a wide range of motivations, including those that defy analysis. Perhaps the author is motivated by status and recognition, or the pure joy of creation or learning. Maybe they want to make something that spreads and see what happens, just to make their mark on reality.

Finally there is The Collaboration, where a group of individuals or companies get together to develop and maintain an open source project. The motivation is primarily the economic value of running the code and the cost savings of developing it together instead of individually or by paying a third party.

So the three types are separated by how the authors capture value from the project or intend to capture value. Of course a single project may have elements of all 3 types. Pocketbase is more or less written by one person and it looks a lot like an Art project with elements of a Collaboration. The evidence of it being a Sales Funnel is minor.

If this news letter were open source, it would primarily be a Sales Funnel. I do enjoy writing and spreading ideas purely for its own sake, but I have limited time to spend on hobbies. Committing myself to writing once a week on LinkedIn requires a financial justification.

The most outrage occurs when a project giving the appearance of being a Collaboration or an Art piece suddenly flips to a non-open source license. Usually though this happens with the projects that are clearly a Sales Funnel for a SaaS product. There is also upset when a project which nominally looks like Art or a Collaboration is revealed as being a Sales Funnel. Such is the case when the core developers get hired or acquired and the new owner puts their stamp on the project.

As a user of Open source or a contributor you have an interest in knowing what really motivates a project’s development. In absolute terms I have no problem with any of the three types. However I think it is useful to identify the type of project one is dealing with and whether that is suitable for your situation. Also whether the project is likely to remain in a particular category or if their is a mismatch between contributor motivations and the outward appearance of the project.

Clearly most projects don’t neatly fit into one of the thread categories. I’ve seen strong evidence of all three in the Linux kernel for example. It’s clearly a Sales Funnel for the Linux Foundation and companies like SUSE. Meanwhile Meta and Netflix collaborate on it because it costs them less than developing their own individual kernels. Finally there are people who contribute just for the sake of taking part.

In terms of attribution I think some of these ideas came from Ron Evans (https://www.linkedin.com/in/deadprogram/) speaking on a podcast about TinyGo although I’m not saying he would agree with any of this, the rest is from Austrian economics and my imagination.