Jan's Blog

Retro Cube

2026-01-24T00:00:00+00:00
A while ago, I became absolutely fascinated with cyberdecks. On my journey to design and build my own, I started small, and this is the result: Your browser does not support the video tag. </video>
It's a 3D-printed housing for a Raspberry Pi Zero W 2, an SSD1309 OLED display, and a rotary encoder for navigation.
Before this, I used Blender to design my custom 3D-printed things</a>, simply because I'm familiar with Blender</a> and was a little intimidated to learn CAD. But then I discovered FreeCAD</a> and thought it would be a good time to actually try it. It was so worth it. It took about two days, and I won't go back to Blender.
Features</h3>
The Retro Cube has three views. You can switch between them by rotating the dial. If you press it down, the display turns off completely, which can be useful at night. The three views are:

Clock</li>
Weather dashboard</li>
Message</li> </ol>
The first two are rather basic, but the last one is the most interesting. There is a server running at retro-cube.jflessau.com</a> serving a basic-auth protected endpoint that returns a text message.
The Pi fetches that text at a configurable interval and displays it in the respective view. The second endpoint of the server is a web form where users can alter the message.
Soooooooo
If you want to say hi or something else, go to retro-cube.jflessau.com</a>, log in with
`user: retro password: cube </code></pre> and send something :)And in case you want to build your own Retro Cube, head over to the repo</a> for the code, STL files for printing, and a list of parts.`



Gasoline Price Cycles
2025-12-23T00:00:00+00:00
Gas stations in Germany are required by law to send their prices to a central database, maintained by the German Cartel Office. Regular people like you and me can apply to access the data for the purpose of building websites or apps to help consumers find cheap gas stations.</p>
A while ago a friend and I did just that and built https://mcsprit.de</a>.

There are many of those services; the Cartel Office lists</a> them on their website.</p>
While working with the data, I found the following:</p>
</p>
A chart of average prices in euros for Diesel, Super E5 and Super E10 of all 15075</code> gas stations in Germany from 2025-12-21 5:53</code> to 2025-12-21 21:47</code> at 10 minute intervals.</p>
There are sudden increases followed by gradual decreases in the prices of all three fuel types, and I wondered why.</p>
Demand</h2>
My initial thought was: The first spike is caused by early commuters, starting to work at 7:00, then a second wave starting at 8:00, and so on; elevated demand causing prices to rise.</p>
But I would expect a more wavy pattern then, not these sudden jumps. Also, there are too many spikes. The amount of cars on the road does not really spike, I thought.</p>
To check that, I downloaded a dataset from Hamburg's Transparency Portal</a>. Hamburg publishes data of infrared traffic sensors with a resolution of 15 minutes. Here is the average amount of cars passing a sensor per 15 minutes from all 52 active sensors plotted for 2025-12-21</code>:</p>
</p>
I am using the volume of cars on the road as a proxy for demand at gas stations here. The curve does not show many sudden jumps, with an exception at 20:00. To be fair, this is car volume in Hamburg only, not nationwide. But fuel prices in Hamburg alone show the same pattern as the nationwide average. This suggests that, at least in Hamburg, the price patterns are not correlated with demand.</p>
Supply</h2>
Gas stations have to buy their fuel at some price, which I thought could vary a lot over the day. The German Cartel Office says:</p>

After the oil has been processed in refineries, the oil products are sold on wholesale markets mainly on the basis of long-term supply contracts</strong> (term contracts); the volumes traded on the spot market are much smaller.
Press Release on Sector Inquiry on Refineries</a></cite></p>
</blockquote>
Gas stations mostly buy fuel based on long-term contracts. And those long-term</em> contracts are pretty long:</p>

In detail, the inquiry found that well over 50 per cent of all volumes traded at the wholesale level, irrespective of the product traded, are based on term contracts, which typically have a term of one year.
Sector inquiry 2025, Cartel Office</a></cite></p>
</blockquote>
The contracts might last a year, but the price gas stations pay for fuel...</p>

...is based on the price
assessment provided at the contractually agreed point in time (e.g. the time of supply) or on the
average prices over a specified period (usually the month or week of supply)
Sector inquiry 2025, Cartel Office</a></cite></p>
</blockquote>
So prices change on a monthly or weekly bases. If the price assessment is based on delivery, then the price changes as often as the gas station gets deliveries. Some stations get multiple deliveries per day, but I doubt they are as frequent or synchronized</em> across stations as the price changes in the chart.</p>
From what I can tell, supply prices are unlikely to be the cause of the price pattern either.</p>
Price cycles</h2>
If supply and demand are not correlated with the price pattern, then what is?</p>
The best theory I've found to explain this particular pattern is described in the Competitive price cycles paper</a>. They specifically look at diesel prices in Germany in 2019. It's a derivative of the well-known Edgeworth price cycle</a> theory.
Bottom line, very broadly, is this:</p>
Some customers are price sensitive and buy at stations with lower prices, others are more brandloyal and shop at their preferred station, even if prices are a little higher. From here it goes like this:</p>

To get more nonloyal customers, ALL stations lower prices</li>
Prices lower further over time, with every station trying to undercut the others</li>
At some point prices have to rise, otherwise stations make no profit</li>
Stations with loyal customers raise prices first and drastically, but not so much that they lose loyal customers</li>
Other stations follow quickly, as their current prices are way too low compared to competitors</li>
Repeat</li>
</ul>
The key points here, I think, are that bigger brands with more loyal customers initiate the price increases. There is no point in ramping up slowly, because most other stations will still be cheaper anyway. So they do it quickly and drastically. And they can do so because they can be somewhat sure customers loyal to them will stay.</p>
Other gas stations then follow very quickly, leaving little time for consumers to benefit from the lowest prices. This raises the question...</p>
Where do gas stations get their competitor's prices from?</h3>
The theory above assumes that gas stations know their competitors' prices in (almost) real time.</p>
Consumers are using the fuel price apps to find cheap stations. It would be naive to think that gas station owners don't use these services as well. In fact, the central database maintained by the Cartel Office is, in my opinion, the only practical way to get near-real-time prices of competitors.</p>
Earlier, we looked at the average prices across all gas stations in Germany, but the pattern is visible at individual stations as well:</p>
</p>
This GIF shows the prices of two individual stations on the same day (one frame per station). The stations are on the same road, about 5 km apart. One is a JET station, the other is an OIL! station. As far as I know, JET and OIL! are not part of the same company. There is also no clear line of sight between the two stations. They can't see each other's price boards. Yet, their prices are pretty much in sync down to just a couple of minutes.</p>
This is not an isolated case. See for yourself by picking pairs of stations in one of the price apps listed</a> by the Cartel Office.</p>
Are gas stations allowed to use the data?</h3>
Disclaimer: I'm not a lawyer.</p>
To access the real-time data, you have to apply at the Cartel Office. You have to use your access to build some kind of publicly available price app or website for consumers.</p>
The Cartel Office can revoke access to the data for price apps if they find that data is provided to, say, mineral oil companies. Aside from that, there seems to be no kind of punishment for either services providing the data to mineral oil companies or mineral oil companies using the data.</p>
So mineral oil companies cannot directly access the data nor apply for access themselves. But they can get the data from the same price apps consumers are using. Since the apps are required to be public, there is little to prevent gas stations from using them to get competitor prices.</p>
Note that applying for access to the data is completely free, yet a bit of a bureaucratic hassle</a>.</p>
To be clear, I strongly believe the Cartel Office has this database for the sake of consumers. Station owners accessing the data is just inevitable.</p>
Are consumers getting more transparency?</h2>
Thanks to the central database, prices are very transparent. But by now they change so frequently that it is questionable if consumers really benefit. How useful is it to know the price at a gas station if it might change again before you get there?</p>

While prices were changed around four to five times a day in 2014, the number of price changes already reached an average of 18 times a day in early 2024 and is still on the rise.
Press Release on Sector Inquiry on Refineries</a></cite></p>
</blockquote>
What should consumers do?</h2>
Price differences between stations at any given moment are still relevant and differ by several cents per liter. For example, stations near highways tend to be much more expensive.</p>
Using an app to find a station that's cheaper right now</em> is still useful. But actually getting the exact price an app shows you is unlikely when refueling during these price cycles. More so when you have to drive a while to get to the gas station.</p>
If you're feeling lucky, you can try to time your refueling just before a price spike. If you can wait, refuel later in the day. And if you have time and want to play safe, find a cheap station at night. Most stations are pretty expensive at night, but if you find a cheap one, it's also likely to stay cheap for a while:</p>
</p>
The chart shows fuel prices of one station for just over 24 hours. Note that there are no changes at all between 2025-12-20 22:00</code> and 2025-12-21 05:00</code>.</p>
Regulation</h2>
With a resolution passed on 21 November 2025</a>, the German Bundesrat (Federal Council) calls on the federal government to examine how petrol prices can be made more transparent for consumers, specifically by limiting the number of price increases per day.</p>
It is not safe to say that limits on price changes would benefit customers after all. Stations may set prices a bit higher by default to account for less frequent opportunities to increase them later. But customers would at least have a more stable price environment to make decisions in.</p>
Austria implemented such limits</a>, allowing for just one price increase per day, exactly at 12:00. Price decreases are allowed at any time. As a result, the best time to refuel in Austria is just before 12:00, while prices after that are probably higher than they would be without the regulation.</p>
Conclusion</h2>
Real-time fuel price data can be used by consumers and</em> gas stations to their advantage. The situation for customers is not ideal as prices change often and in sync with other stations. The price you see in an app might not be the price you pay when you get there. And since most stations' prices rise and fall together, looking out for alternative stations is less useful, since all are either up or down at the same time.</p>
Putting regulation in place to limit price changes could help, but designing such regulation is not trivial.</p>
Opinion</h2>
Since this is, after all, my personal blog, here is my very personal opinion on the matter:</p>
Introducing the central database for fuel prices was a good idea. I support gathering non-personal data that was very public anyway, just not centralized and therefore hardly useful.</p>
Aforementioned price cycles are a known phenomenon in economics and have been for many decades. Economics is not my strong suit, I still think this outcome could have been predicted to some extent. Politics, also not my strong suit, could have put limits on price changes into the same piece of legislation that introduced the requirement to report prices to the Cartel Office. Arguing for this would have been reasonable back then, when price changes were way less frequent anyway.</p>
The second-best time to push for limiting price changes would be now. We got the data, we see the effects. Prices are transparent, yet of little use to customers if they are altered within minutes. It might be unclear how to exactly design the limits, but other countries have done it, let's learn from them. The current situation is clearly not ideal for consumers.</p>


sqlx-fmt
2025-11-08T00:00:00+00:00
I'm a huge fan of SQLx</a>, and it has always bothered me that SQL within its macros isn't formatted automatically:</p>
sqlx</span>::</span>query</span>!</span>(
</span>    </span>r</span>#"
</span>        select *
</span>        from users
</span>        where verified = true
</span>    "#
</span>)
</span></code></pre>
So I created sqlx-fmt</a>, a CLI and GitHub Action that formats SQL within SQLx macros in Rust code. It uses sqruff</a> as a formatter under the hood. Tree-sitter</a> is used to parse the Rust code and extract the (raw) string literals within SQLx macros. I had used regexes initially, but that required a lot of edge case handling, which tree-sitter solves elegantly.</p>
Installation</h2>
Clone the repo</a> and install sqlx-fmt using cargo:</p>
# install sqlx-fmt
</span>git</span> clone https://github.com/jflessau/sqlx-fmt.git
</span>cd</span> sqlx-fmt
</span>cargo</span> install</span> --path</span> .
</span></code></pre>
If you haven't already, install sqruff</a>, which can also be installed via cargo:</p>
cargo</span> install sqruff
</span></code></pre>
Usage</h2>
This CLI has two commands: format</code> and check</code>.</p>
# format code
</span>sqlx-fmt</span> format</span> --path</span> path_to_files
</span>
</span># check if code is formatted
</span>sqlx-fmt</span> check</span> --path</span> path_to_files
</span></code></pre>
Configuration</h3>
Optionally, specify a sqruff config file</a> with --config .sqruff</code>. The config file allows you to specify things like SQL dialect, rules to apply, and indentation style. Here's an example .sqruff</code> config file:</p>
[</span>sqruff</span>]
</span>dialect </span>= </span>postgres
</span>rules </span>= </span>ambiguous,capitalisation,convention,layout,references
</span>
</span>[</span>sqruff</span>:</span>indentation</span>]
</span>indent_unit </span>= </span>space
</span>tab_space_size </span>= </span>4
</span>indented_joins </span>= </span>True
</span></code></pre>
GitHub Action</h2>
You can also use it as a GitHub Action</a> to automatically run format checks in CI:</p>
steps</span>:
</span>  - </span>name</span>: </span>checkout code
</span>    </span>uses</span>: </span>actions/checkout@v4
</span>
</span>  - </span>name</span>: </span>run format checker
</span>    </span>uses</span>: </span>jflessau/sqlx-fmt@main
</span>    </span>with</span>:
</span>      </span>context</span>: </span>"./code_to_format"
</span>      </span>config-file</span>: </span>"./code_to_format/.sqruff"
</span>      </span>fail-on-unformatted</span>: </span>"true"
</span></code></pre>


Retweet 2 Win
2025-10-05T00:00:00+00:00
Back when I was studying, around 2015, I saw this post</a> from a guy winning stuff on Twitter with a self-made bot. I couldn’t find the code, so I wrote my own.</p>
On Twitter, at least at the time, people and companies used giveaways to promote their accounts. They asked for retweets and, in return, promised to randomly select a winner from all the retweeters to receive some kind of prize.</p>
Hurdles</h2>

Twitter enforced rate limits, so searches and retweets were limited.</li>
Some giveaways required you to also follow the account, and following too many people too quickly would get your account blocked.</li>
Surprisingly, there were other bots tweeting tweets with keywords like #retweet2win</code> without actually offering a prize.</li>
</ol>
The first two were relatively easy to solve: just add delays and unfollow accounts after some time. That made the whole process more time-consuming, but since the number of giveaways was also limited and spread out over time, that was fine.</p>
One day I found that one of those anti-giveaway-bot bots had my bot's name plus some suffix as its own name. So someone must have seen my bot and created an account to interfere with its mission.</p>
To avoid wasting time retweeting those bots, I made a blacklist of accounts not to interact with. After a few days, I found most of these bots and also some scammy giveaway accounts I had no interest in retweeting.</p>
Scams</h2>
Every other day, I would check my DMs to see if any giveaway accounts had selected me as the winner. Pretty straightforward, but, of course, most of these alleged wins were things like:</p>

“You won the PS4! I only need you to send me 20 € on PayPal for shipping, then I’ll send it to you.”</p>
</blockquote>
To get real, physical prizes sent to my door, I needed to disclose my address to the giveaway creator. So I spent quite some time figuring out if the whole giveaway was a scam or not. That was a bit annoying.</p>
Prizes</h2>
I won random stuff like:</p>

tickets to expos I’d never heard of</li>
gift cards for online shops</li>
an educational poster for geography classes</li>
self-published books</li>
plushies</li>
</ul>
My favorite to this day is a Corgi certificate</em>: a piece of relatively thick paper with a corgi on it, surrounded by the phrase “100% woof”. I don’t own a dog and have no clue what this certificate certifies. ¯\(ツ)</em>/¯</p>
For stuff I couldn’t use myself (tickets to concerts in other countries, etc.), I found people on Reddit who were interested so the tickets wouldn't be wasted.</p>
Customer Support</h2>
I made the bot open source and, after some days, had a few people emailing me asking for instructions to set it up or reporting bugs. That felt nice, finally seeing other people take interest in what I’d created. But it turned out to be quite time-consuming. And I also didn’t want armies of my bots polluting Twitter. Just, you know, mine and maybe a handful of others.</p>
This taught me a valuable lesson: If you put some code into the world, you basically lose control over it.</p>
After a few weeks, requests for support ended and I eventually archived the GitHub repo.</p>
Regrets</h2>
Once a few physical wins arrived at my doorstep, I told a friend about my bot. He wanted to try it, and in an afternoon, we set it up on his Raspberry Pi.</p>
Within a month, he won a 3D printer, surpassing pretty much everything I had won up to that time and, in hindsight, ever. Envy is an ugly emotion and it happened almost a decade ago, but it still stings somehow.</p>
Looking back at this today, I wouldn't do something like this again. Who really likes bots on their beloved social media platform? And I won giveaways other people, potentially much more interested in the prizes, could have won. So, yeah, regret.</p>


Ritmo
2025-10-04T00:00:00+00:00
Yes, yes, another habit tracker. But this one is different. Well, not really. But it's mine.</p>
Out of frustration with previous attempts at using other trackers, I decided to build my own. Here are its key features:</p>

No login required</li>
Works offline</li>
Import & export data</li>
PWA-ready</li>
MIT licensed</li>
</ul>
GitHub Repo: https://github.com/jflessau/ritmo</a>

Live Demo: https://ritmo.jflessau.com</a></p>
I've been dogfooding Ritmo for half a year now as a PWA on iOS and have never looked back.</p>
Screenrecording</h2>

  
  Your browser does not support the video tag.
</video>


Sitcom as a Service
2025-10-04T00:00:00+00:00
A while back, I created a web app that fills the silence in conversations with canned laughter using machine learning. This, along with my multiplayer bullshit bingo</a>, turned out to be slightly more fun than I expected while building it.</p>
You might know the rush of building something and how convinced you are that it will be great, only to find out later that it is not. Not this time, so I’m keeping it around for people to try: https://canned-laughter.jflessau.com</a>.</p>
Hint: It won't work in mobile Safari for reasons I'm too lazy to investigate.</p>
</p>


Coding with AI
2025-08-26T00:00:00+00:00
I code with AI. Autocomplete is, in my opinion, the most useful AI coding tool. Typing the first two characters of a line of code and having the rest filled in for you is a game changer. Even if it only works 50% of the time, it saves a substantial amount of keystrokes and time. But I admit, it's a distraction in the typing flow. Because of that, my typing skills have suffered. Even more concerning: When I’m on the train or at any place without internet, having no access to copilot, I feel much slower than I was before using it at all.</p>
It's worth it, at least for now. It saves me countless hours of looking up the same things in docs I keep forgetting (how to parse json from a file, how to configure a reqwest client, how to set up tracing, how to start postgres with testcontainers, etc.).</p>
Agentic Editing</h2>
Agentic editing is another story entirely. Witnessing how the Claude CLI burns through tokens and creates a functional little web app that does not look too bad is mesmerizing. Heavily using git and branching is a good idea here and makes this whole process feel even faster, creating branches left and right, discarding some and progressing with others.</p>
At some point, I found, it breaks apart. Prompts do not hit the mark, bugs stay or reappear, and the code structure gets messy. By then, the amount of code produced is already substantial. And the hard part is now reading and understanding it in order to carry it over the finish line. For some smaller projects, AI gets it over there alone, but if not, there is a lot of work to do.</p>
The work is mostly reading and understanding the code, then fixing not only the parts that do not work but also the structure so that it fits your mental model more. So that it becomes more like your</em> code.</p>
Agentic editing still has some use cases in my workflow. I mainly use it to speed up setups and confined tasks that require context from multiple files. Here are some examples:</p>

Looking at multiple files and an error message to help me debug</li>
Setting up a node project, pretty much eliminating the need for templates. I just say I want, for example: TypeScript, Tailwind, SolidJS, unplugin icons with autoimport, and Bun.</li>
Writing a down-migration file for postgres based on an up-migration or generating structs for each table.</li>
</ul>
Aside from generating code, I use the agentic features in my editor of choice for getting to know a new codebase by asking questions about it and getting mostly accurate responses with references to files and lines in the code at hand.</p>
Conclusion</h2>
Coding with AI can make developers faster and reduce frustration caused by constantly googling for docs and references. I think the hard limit on how fast we can be is actually understanding the codebase. Don’t push to prod until you do. AI can help you understand, but at some point, the code has to pass through wetware</a>. Therefore, I find the (mostly) line-by-line autocomplete much more useful than agentic editing. It does work for you in chunks you can follow and understand while saving time on typing.</p>


InkDay
2025-05-17T00:00:00+00:00
I made a 3D printable frame for a 7.5" e-ink display and a website to control its content.</p>
On the website, you can pick one image for each day via a calendar interface. A Raspberry Pi connected to the display fetches the current day's image from the website and displays it on the e-ink screen.</p>
Demo</h2>
The frame on my desk is controlled by a public version of this website. You can control what I see here: https://inkday.jflessau.com</a>.</p>
Everything You Need to Build It</h2>
You can find the code, 3D file, list of the parts used, and instructions for assembling them in this GitHub repo:</p>
👉 InkDay GitHub Repo</a></p>
More Pics</h2>
Back</h3>
</p>
Website Screenshot</h3>
</p>


Beam Base
2025-05-07T00:00:00+00:00
This cylindrical base for RPG minis has a built-in field of view</strong> (FOV) cone that can be adjusted to different angles. The FOV is projected onto the table, making it easy to see what the mini can see.</p>
I find the FOV features of tabletop simulators like Roll20 useful, so I wanted to create something similar for real-life use.</p>
It's made of three printable parts and a light source. Here are the STL files:</p>

Base</a></li>
Slider</a></li>
Top</a></li>
</ul>
I used these balloon lights</a> as light sources. You can use something else as long as it fits into the rails of the bottom part (w: 10 mm, h: 10 mm, d: 40 mm) leaving some space in the depth direction to allow the adjustable beam angle to function properly. You might need some adhesive to glue the light to the slider. Sticky Tack, poster putty, or Patafix should do the trick.</p>
Pictures</h2>
Individual Parts</h3>
</p>
Narrow Beam</h3>
</p>
Wide Beam</h3>
</p>


Postgres Ultras T-Shirt
2025-02-18T00:00:00+00:00
I looked around at all the known sites selling shirts but cound't find one for Postgres that I liked. So I made my own.</p>
Download a high-res PNG below from here</a>.</p>
</p>


SurrealDB
2025-01-02T00:00:00+00:00
Are you the relational type when it comes to databases? Do you like Postgres, have heard of SurrealDB</a>, and want to know if it's worth a shot? This post is for you :)</p>
What is SurrealDB?</h2>
SurrealDB is a multi-modal database. You can store relational data, graphs, documents, and time series in it. It supports full-text search, vector search, has its own SQL-like query language, and lets you store and execute ML models. You can have fine-grained access control (down to fields, aka columns), which leads to their claim of being a backend as a service (think Firebase</a>).</p>
That's quite a lot cover. I’ll focus on relational data storage, consistency, querying, performance, and DX. Here we go.</p>
Data Consistency</h2>
Strict Mode</h3>
You can start SurrealDB either with or without the strict</code> flag. Without</strong>, you can create records (in Postgres, these would be rows) without defining their schema. That might be cool for prototyping, but it's not my cup of tea. Note that strict mode is either on or off for the entire database instance.

Here I create a dog</code> record without defining a schema for it:</p>
-- insert record
</span>create dog content {
</span>    name: "cookie",
</span>    good_dog: true
</span>};
</span></code></pre>
With</strong> strict mode, you need to define what</em> records exist (e.g. dog</code>), but not necessarily how they look:</p>
-- define table
</span>define table dog schemaless;
</span>
</span>-- insert record
</span>create dog content {
</span>    name: "cookie",
</span>    good_dog: true
</span>};
</span></code></pre>
Schemafull & Schemaless</h3>
We defined a schemaless table for dog</code>, and aside from actually putting records in that specific table, they can have pretty much any shape or form we want.</p>
create dog content {
</span>    title: "sir doggo",
</span>    age_years: 3
</span>};
</span></code></pre>
By the way, all records have IDs, which are generated automatically if not explicitly specified.</p>
We can</em> define fields for schemaless tables. Inserting with extra fields is acceptable (and they become part of the stored record), but inserting with missing fields returns an error:</p>
define table dog schemaless;
</span>define field name on dog type string;
</span>define field age_years on dog type int assert $value >= 0;
</span>
</span>-- this insert works!
</span>create dog content {
</span>    name: "cookie",
</span>    age_years: 2,
</span>    extra_field: "good dog"
</span>};
</span>
</span>-- doesn't work, because "age_years" is missing
</span>create dog content {
</span>    name: "cookie"
</span>};
</span></code></pre>
Now we're done prototyping and know how the schema should look. We discard the old table and define it again, this time as schemafull</strong>. Trying to insert records with missing or extra fields will return an error for schemafull tables.</p>
remove table dog;
</span>
</span>define table dog schemafull;
</span>define field name on dog type string;
</span>define field age_years on dog type int assert $value >= 0;
</span>
</span>create dog content {
</span>    name: "cookie",
</span>    age_years: 2
</span>}
</span></code></pre>
Altering the Schema</h3>
One thing I really don't like about SurrealDB: If we change, say, a field's type from int</code> to string</code>, I would expect all records in that table to conform to the new schema. However, they don't.</p>
Here, I use the overwrite</code> keyword (there is an alter</code> keyword, but it works for tables only, at least for now) to change the field type of zip_code</code> from int</code> to string</code>. However, existing records still have the old type for that field. You must manually update the records to fit the new schema, even in strict mode for schemafull tables 👎</p>
-- define table
</span>define table address schemafull;
</span>define field street on table address type string;
</span>define field zip_code on table address type int;
</span>
</span>-- insert record
</span>create address content {
</span>    street: "Second Avenue 2",
</span>    zip_code: 1234
</span>};
</span>
</span>-- change field type
</span>define field overwrite zip_code on table address type string;
</span>
</span>-- add field
</span>define field test on table address type string;
</span>
</span>-- remove field
</span>remove field street on table address;
</span>
</span>-- query records
</span>select * from address;
</span></code></pre>
Querying the addresses with select * from address;</code> returns:</p>
[
</span>	{
</span>		id: address:o5fabxa3eir08n731b5o,
</span>		street: 'Second Avenue 2',
</span>		zip_code: 1234
</span>	}
</span>]
</span></code></pre>
Nothing changed. Both street</code> and zip_code</code> (as string) are still there and the new field test</code> is missing as well.</p>
Altering the schema doesn't alter old data. New records will have the new schema, though.</p>
This is especially dangerous, I think, when you've just added a non-optional field to a table that your backend code relies on to be neither null nor undefined. Older records won't have that field until you write a query to update them. That was a footgun, at least for me.</p>
Let's transition to something more harmonious: Relations!</p>
Record Links</h3>
There are two main ways to create relations between records. Let's explore the simplest one: Record Links</strong>. First, we need another table: human</code>. Then we define a field of type record<dog></code>. I find that pretty neat.</p>
define table human schemafull;
</span>define field name on human type string;
</span>define field loves on human type record<dog>;
</span></code></pre>
But there is no guarantee that the dog referenced in the human table actually exists. Hm...</p>
Relations</h3>
We could create custom events that fire on create/update/delete, but who wants that? Another way to relate two records is through a Relation</strong>. Relations are basically tables, but with two default fields: in</code> and out</code>, both holding a record ID. You can add more fields to describe the relation.</p>
-- create records for human and dog
</span>create human:human_1 content { name: "alex", loves: dog:dog_1 };
</span>create dog:dog_1 content { name: "cookie", age_years: 2 };
</span>
</span>-- define relation between human and dog
</span>define table cares_for type relation in human out dog enforced;
</span>define field since on cares_for type datetime;
</span>
</span>-- create relation
</span>relate human:human_1 -> cares_for -> dog:dog_1 content { since: time::now()}
</span></code></pre>
Relations also won't guarantee that their related records exist—not by default. The enforced</code> keyword used in the relation's definition will prevent the insertion of relations with missing records in referenced tables. If a referenced record is deleted, the relation will be deleted as well.</p>
You would need to build custom events to achieve the "on delete cascade" feature that Postgres offers. However, a PR</a> to go in that direction was opened less than a week ago. Fingers crossed 🤞</p>
Data Storage</h2>
SurrealDB stores data in key-value stores. This is very</strong> important to know, I think. You can choose between RocksDB, FoundationDB, IndexedDB, SurrealKV, TiKV, or an in-memory KV. SurrealKV is still in beta, though.

TiKV</a>, for instance, is horizontally scalable while being ACID compliant.</p>
Record IDs follow the format table_name:id</code>. Record IDs are used to retrieve records from KV stores quickly. SurrealDB strongly encourages you to use these record IDs for filtering whenever possible because they are indexed by default. And you really better do that, or else...</p>
Performance</h2>
Performance can be a problem. As far as I know, SurrealDB hasn't published any benchmarks, which is sus. So I measured some things myself.</p>
Test: Query Table with 1 Mio. Records</h3>
I populated a user</code> table with one million rows in both SurrealDB and Postgres for some query benchmarks. I'm using SurrealDB 2.1.2 with the in-memory KV store and Postgres 17.0, both running in Docker on an M1 Mac.</p>
SurrealDB</h4>
-- define table
</span>define table user schemafull;
</span>define field id on table user type string;
</span>define field username on table user type string;
</span>define field email on table user type string;
</span>define field password_hash on table user type string;
</span>define field coins on table user type int;
</span>
</span>-- populate table (took 65 s)
</span>FOR $i IN 1..=1000000 {
</span>    create user content {
</span>        username: string::concat('username' + return <string> $i),
</span>        email: string::concat('email' + return <string> $i),
</span>        password_hash: string::concat('id' + return <string> $i),
</span>        coins: rand::int(0, 1000000)
</span>    };
</span>}
</span></code></pre>
Postgres</h4>
-- define table
</span>create table </span>"</span>user</span>" (
</span>	id </span>text primary key</span>,
</span>	username </span>text </span>not </span>null </span>check</span> (length(username) </span>> </span>0</span>) unique,
</span>	email </span>text </span>not </span>null</span>,
</span>	password_hash </span>text </span>not </span>null</span>,
</span>	coins </span>int </span>not </span>null
</span>);
</span>
</span>-- populate table (took 10 s)
</span>DO $$
</span>DECLARE
</span>    id </span>text</span>;
</span>    username </span>TEXT</span>;
</span>    email </span>TEXT</span>;
</span>    password_hash </span>TEXT</span>;
</span>    coins </span>INT</span>;
</span>BEGIN
</span>    FOR i </span>IN </span>1</span>..</span>1000000</span> LOOP
</span>    	id :</span>= </span>'id' </span>||</span> i;
</span>        username :</span>= </span>'user' </span>||</span> i;
</span>        email :</span>= </span>'user' </span>||</span> i </span>|| </span>'@example.com'</span>;
</span>        password_hash :</span>= </span>'hash' </span>||</span> i;
</span>        coins :</span>=</span> (RANDOM() </span>* </span>1000 </span>+ </span>1</span>)::</span>INT</span>;
</span>
</span>        </span>INSERT INTO </span>"user"</span> (id, username, email, password_hash, coins)
</span>        </span>VALUES</span> (id, username, email, password_hash, coins);
</span>    </span>END</span> LOOP;
</span>END</span> $$;
</span></code></pre>
I then queried both databases for:</p>

Number of users</li>
Ten users with the most coins</li>
Sum of all coins of all users</li>
</ol>
My measurements aren't very scientific or exact, but they're still worth a look, I think.
Keep in mind: Postgres has been around for a long time. SurrealDB is relatively young, and performance optimizations could be coming. They could</em> be huge. Or not. I don't know.</p>
Results SurrealDb</h4>
-- count all users ≈ 6 s
</span>select count() from user group all;
</span>
</span>-- get ten users with most coins ≈ 8 s
</span>select * from user order by coins desc limit 10;
</span>
</span>-- sum of all coins from all users ≈ 8 s
</span>select math::sum(coins) from user group all;
</span></code></pre>
Results Postgres</h4>
-- count all users ≈ 38 ms
</span>select </span>count</span>(</span>*</span>) </span>from </span>"user"</span>;
</span>
</span>-- get ten users with most coins ≈ 30 ms
</span>select </span>* </span>from </span>"user" </span>order by</span> coins </span>desc limit </span>10</span>;
</span>
</span>-- sum of all coins from all users ≈ 25 ms
</span>select </span>sum</span>(coins) </span>from </span>"user"</span>;
</span></code></pre>
As you can see, Postgres is orders of magnitude faster than SurrealDB for these queries.</p>
Indexes to the Rescue?</h3>
What about indexes? Can't they help with, say, ordering by coin amount? Creating an index for the coins</code> field and re-running the query for the ten richest users reduced the time for that query by ... nothing?. It still takes about 8 seconds</code> in SurrealDB. I'm not sure if I did something wrong, but I expected a little speedup:</p>
-- define index ≈ 26 s
</span>define index coins_index on table user columns coins;
</span>
</span>-- get ten users with most coins ≈ 8 s
</span>select * from user order by coins desc limit 10;
</span></code></pre>
Adding an index for email</code> and querying for two users with an email containing email9999</code> drops the time from approximately 8 seconds</code> to about 50 milliseconds</code> when using an index. So indexes can</em> help you:</p>
-- select without index ≈ 8 s
</span>select email
</span>from user
</span>where email contains "email9999"
</span>order by email desc
</span>limit 2;
</span>
</span>-- define index ≈ 25 s
</span>define index email on table user columns email;
</span>
</span>-- select with index ≈ 50 ms
</span>select email
</span>from user
</span>where email contains "email9999"
</span>order by email desc
</span>limit 2;
</span></code></pre>
Local vs. Cloud</h3>
And for good measure, I did the exact same measurements on a DB instance in SurrealDB's cloud. Just in case my way of running the DB locally was the bottleneck. Here is a summary of all results</p>
Postgres local</h4>
Query</th> Time</th></tr></thead>

populate table</td> 10 s</td></tr>
count all users</td> 38 ms</td></tr>
get ten users with most coins</td> 30 ms</td></tr>
sum of all coins from all users</td> 25 ms</td></tr>
</tbody></table>
SurrealDB local</h4>
Query</th> Time</th></tr></thead>

populate table</td> 65 s</td></tr>
count all users</td> 6 s</td></tr>
get ten users with most coins</td> 8 s</td></tr>
sum of all coins from all users</td> 8 s</td></tr>
</tbody></table>
SurrealDB cloud (free tier)</h4>
Populating the table worked, but i got The engine reported the connection to SurrealDB has dropped</code> for the rest of the queries. Every time. It might be that the free tier is too slow for these queries.</p>
Query</th> Time</th></tr></thead>

populate table</td> 339 s</td></tr>
count all users</td> ?? s</td></tr>
get ten users with most coins</td> ?? s</td></tr>
sum of all coins from all users</td> ?? s</td></tr>
</tbody></table>
SurrealDB cloud (paid "large" instance)</h4>
This instance has 2 vCPUs</code>, 8 GB RAM</code>, and 64 GB storage</code> and costs $0.301/hour</code> or about $215/month</code>.</p>
Query</th> Time</th></tr></thead>

populate table</td> 48 s</td></tr>
count all users</td> 3 s</td></tr>
get ten users with most coins</td> 6 s</td></tr>
sum of all coins from all users</td> 4 s</td></tr>
</tbody></table>
Bugs</h2>
If you want to use SurrealDB in production, I recommend first skimming through their GitHub issues labeled with "bug"</a>. That might sober you up before jumping on the hype train too quickly.</p>
This one</a> kept me busy for a while: SELECT * FROM users WHERE email CONTAINS "@example.com"</code> returned no results if email</code> had an index.</p>
Another, similar issue</a> I ran into also altered a SELECT query’s result if an index was present versus not. But it got resolved a few days after I published this post. Thanks to @mithridates on Bluesky for letting me know :)</p>
I have little doubt that the SurrealDB team will fix these bugs and others like them. But for now, they are there and you should be aware of them.</p>
Developer Experience</h2>
As demonstrated in this post, Hugo</a>, my SSG of choice, has syntax highlighting for SQL but not for SurrealQL. At least for now.</p>
But there are syntax highlighting extensions for VS Code, Vim and Zed</a>, the editors I'm working with.</p>
SDKs</h3>
SurrealDB has quite a few SDKs</a> for different languages. I've worked with the ones in Rust and Typescript. The one in Rust is fine, as is the one in Typescript.</p>
But using the latter revealed a rather unpleasant side effect of record IDs in SurrealDB: As mentioned, they are composed of the table's name and the record's ID (table_name:record_id</code>), e.g. user:1234</code>. When querying the DB, it returns an object where the IDs are of class RecordID</code>. Additionally, that class cannot be serialized to JSON. So you have to use the utility method jsonify()</code> on on objects retrieved from the DB before you can, for example, send them as JSON to a client, who called your REST API.</p>
The client gets user:1234</code> as a string and needs to work with it. Suppose it wants to generate a link to the profile page of a user. Unless you manually remove the table part from the record ID, a link would look like this: /users/user:1234/profile</code>.</p>
As far as I know, their SDK doesn't offer a method to convert a string back to a RecordID</code>, which has methods for safely extracting the table and ID parts. Doing that manually might be trivial for record IDs like user:1234</code>, but the ID part could also be an array or object</a>.</p>
Surrealist</h2>
Surrealist is the GUI for SurrealDB, and it is neat. It lets you manage connections to many DBs, query them, visualize their content, and it even paints you a pretty view of your relational data model. Here you can see the Record Link loves</code> and the Relation cares_for</code> between human</code> and dog</code> that we created earlier:</p>
</p>
Conclusion</h2>
I'm excited about this relatively new piece of DB technology!
The feature list is genuinely impressive. Here are three more that I find particularly interesting:</p>

Live queries actively pushing data to the client when new records pop up</li>
Make HTTP calls from within the query language</li>
Run SurrealDB in WASM in the browser</li>
</ul>
Having one DB to store all your data (graph, relational, time series, and more) is compelling. However, I wouldn't use it in production until it gets quite a bit faster, resolves pressing index issues, and offers more convenience for data consistency, like on delete cascade</code>, and an alter</code> keyword that also alters data, not just the schema.</p>
But it's fine for side projects like these I've built with it so far:</p>

Botto</a> - a general-purpose chat bot for the Matrix protocol</li>
Recipe Robot</a> - an AI-powered web app that extracts ingredients from recipes and matches them with actual products from a grocery store, with prices and everything</li>
</ul>
I'm working on two other projects powered by SurrealDB, and I will keep this blog post updated in case there are any more findings along the way.</p>
Your Turn</h2>
If you want to get hands-on with SurrealDB, I recommend downloading Surrealist</a> and setting up a local DB to play around with. You don't even have to learn the query language first to create tables, records, record links and relations. Surrealist has you covered for that.</p>
Use this docker compose file to start a local SurrealDB instance:</p>
version</span>: </span>"3.8"
</span>services</span>:
</span>  </span>surreal</span>:
</span>    </span>image</span>: </span>surrealdb/surrealdb
</span>    </span>ports</span>:
</span>      - </span>"8000:8000"
</span>    </span>volumes</span>:
</span>      - </span>./db_data/content:/data/
</span>    </span>environment</span>:
</span>      - </span>SURREAL_NAMESPACE=default
</span>    </span>command</span>:
</span>      [
</span>        </span>"start"</span>,
</span>        </span>"--username"</span>,
</span>        </span>"test"</span>,
</span>        </span>"--password"</span>,
</span>        </span>"test"</span>,
</span>        </span>"--strict"</span>,
</span>        </span>"file:/data/database"</span>,
</span>      ]
</span></code></pre>


Drawer Inserts
2024-02-12T00:00:00+00:00
A few weeks ago I found this project</a> from node-dojo on GitHub and it used a feature of Blender</a> that I didn’t know about.</p>
node-dojo uses geometry nodes</strong> to automatically fill a 3D box with other boxes, which could then be 3D printed to serve as drawer inserts. I measured a drawer and tried it, but the bin-packing</a> node-dojo implemented wasn’t quite what I needed for my drawers.</p>
So I built my own thing</a> and explored what geometry nodes could actually do.</p>
Working with Geometry Nodes</h2>
Geometry nodes allow you to edit geometry by configuring and connecting nodes. While you could achieve the same and more by writing Python code, these geometry nodes looked so fun that I had to try them.</p>
It turns out they can do a lot, and the learning curve is okay if you're already a little familiar with Blender.</p>
Don’t get me wrong; it was still a challenge, but a fun one. If you're not careful, your nodes will get messy pretty quickly. While text editors offer automatic formatting, with these nodes, you're on your own. This addon for node arrangement</a> provides some level of automatic formatting, but I found that manual arrangement led to much more readable layouts.</p>
</p>
Geometry nodes work similarly to coding in text: you have access to arithmetic and boolean operations. You can group and collapse nodes and reuse networks with fixed sets of inputs and outputs, much like functions in a programming language.</p>
Though it felt more difficult to achieve my desired outcome compared to coding in text. I found it hard to get a quick overview of what's happening in a specific area. Frames (those dark boxes you see in the image above) helped with grouping and naming things. I wished I had discovered them sooner.</p>
Create your custom drawer inserts</h2>
Download the Blender file from here: https://github.com/jflessau/drawer-inserts</a> and open it.

Now, you can experiment with the configuration options and generate your drawer inserts. The repository's README contains a table explaining all configuration options.</p>
Results</h2>
Here is my first drawer with 3D printed inserts:</p>
</p>


BS Bingo
2024-02-10T00:00:00+00:00
It’s fair to say that there is no shortage of bullshit. Sometimes it may even be hard to escape it. You should still try though.</p>
I built a real-time multiplayer bullshit bingo web app for when there is no way around it. It's a chance to still have some fun, even while knee-deep in it.</p>
You can play it right here, no login required: bingo.jflessau.com</a></p>
How it works</h2>
Choose a public template consisting of 25+ words and phrases or create your own, then hit Play</code>.</p>
Bullshit bingo, if you don't already know it, works just like regular bingo. You get your card with a (usually 5x5) grid of words or phrases. For a marketing meeting, your card could look like this:</p>
</p>
When you hear the buzzword, click on it. Once you get a full row, horizontally, vertically, or diagonally, you've got a bingo!</p>
Below your card is a list of all players and a representation of their cards. Cards may contain the same buzzwords, but they are randomly shuffled for each player.</p>
Use cases</h2>
The obvious ones are presentations, meetings, and other occasions where you expect a ton of buzzwords. But that's just the tip of the turd. Bullshit is to be found everywhere.</p>
My favorite use case so far is public transport. Don’t get me wrong, I’m all for it. And playing a game of whose commute was more interesting</em> makes it a lot more enjoyable:</p>
</p>
Games have no time limit so you and your friends can play one over the course of a year or more.</p>
Tech</h2>
The web app is built with Svelte</a>, which might not be as mature as react, but feels less verbose.</p>
The API is built in Rust. Apart from the boring CRUD stuff, there is a WebSocket connection for sending and receiving game updates in real-time. Postgres holds the game’s state and card templates. The Rust service subscribes to Postgres notifications, which is quite convenient when using sqlx</a> as your DB client:</p>
let mut</span> listener </span>= </span>PgListener</span>::</span>connect_with(pool)</span>.</span>await</span>?</span>;
</span>
</span>listener
</span>    </span>.</span>listen_all</span>(</span>vec!</span>[</span>"fields_update"</span>, </span>"players_update"</span>])
</span>    </span>.</span>await</span>?</span>;
</span>
</span>loop </span>{
</span>    </span>let</span> notification </span>=</span> listener</span>.</span>recv</span>()</span>.</span>await</span>?</span>;
</span>    </span>let</span> game_update</span>:</span> PgGameUpdateNotification </span>= </span>serde_json</span>::</span>from_str(notification</span>.</span>payload</span>())</span>?</span>;
</span>    </span>// do something with game_update
</span>}
</span>
</span></code></pre>
The corresponding part in SQL that sets up the notifier looks like this:</p>
create or replace function </span>game_update_notification </span>()
</span>  returns trigger
</span>  language plpgsql
</span>as</span> $$
</span>  declare
</span>    channel </span>text</span> :</span>=</span> tg_argv[</span>0</span>];
</span>  </span>begin
</span>    perform (
</span>      </span>with</span> payload(game_id) </span>as</span> (
</span>        </span>select </span>new</span>.</span>game_id
</span>      )
</span>      </span>select
</span>        pg_notify(channel,
</span>        row_to_json(payload)::</span>text</span>)
</span>      </span>from
</span>        payload
</span>    );
</span>    return </span>null</span>;
</span>  </span>end</span>;
</span>$$;
</span>
</span>create trigger </span>fields_update
</span>after </span>update</span> on </span>bingo</span>.</span>fields
</span>for each row execute procedure game_update_notification(</span>'fields_update'</span>);
</span>
</span>create trigger </span>players_update
</span>after </span>update</span> on </span>bingo</span>.</span>players
</span>for each row execute procedure game_update_notification(</span>'players_update'</span>);
</span></code></pre>
In this case, the notification's payload contains just the game’s ID. It’s up to the Rust service to then load more info about that game and send it to the players. But you could query as much data as you want within a notification trigger.</p>
Shoutout to the makers of the sqlx crate</a>! It is by far the most ergonomic way to interact with a DB in Rust and pretty much every other language I use. ORMs can be nice, until you debug whatever complicated query they’ve constructed for you.</p>
ORMs offer typesafety you might say, but so does sqlx! You can write good old SQL with it and its macros will ensure that the types line up nicely. Works like magic.</p>
Demo</h2>
Here is a demo of it: bingo.jflessau.com</a></p>
If you find a bug, have a feature request, or just want to host it yourself, here is the MIT licensed GitHub repo: github.com/jflessau/bs-bingo</a></p>


Catenary
2024-01-11T00:00:00+00:00
A while back, I spent a couple of hours on a train and pondered how cool it would be to have a chatroom for the entire train.</p>
Problem</h2>
Being on a train is not as enjoyable as it could be. On longer journeys, you have two options: Keep to yourself and be bored, or attempt to strike up a conversation with someone nearby. The latter is high risk, high reward: You might have a fun talk with a complete stranger, or you could end up with someone less interesting or more annoying.</p>
Solution</h2>
After many months since that train trip, I'm finally bringing this idea to life. Here's how it works:</p>

User enters the site.</li>
The site tracks the user's location for a few seconds.</li>
The site finds other users traveling with roughly the same speed, direction, and current location.</li>
The site displays messages from matching other users.</li>
The site allows users to send and up- or downvote messages.</li>
</ul>
Now, you can talk to people who share your (un)fortunate fate of being on the same vehicle.</p>
Nasty Details</h2>
With this small set of rules, people traveling in the same train, bus, etc., should see each other's messages. Tuning the parameters for matching users is the challenging part. Buses are short and change direction often, while trains are long, fast, and less likely to change direction drastically. I guess this part is learning by dogfooding.</p>
The obvious drawback is that you can't really use the app if you are not moving. However, this is part of the appeal.</p>
Tech</h2>
It not only requires a few real users, but they also need to be moving in the same direction with the same speed at the same time. I have (almost) zero hopes that more than a few dozen people will ever see, let alone try it. So, building it for the sake of it being used is not enough to start building it.</p>
I'm encouraging myself to do it by choosing a somewhat new tech stack: Leptos</a>. Leptos is a frontend framework for Rust with hydration, SSR, and the usual promises of being (blazingly) fast. And so far, I like it.</p>
Chats and their messages are ephemeral; nothing is stored long-term, no database, no login, everyone stays anonymous. I'm imagining active users having a digital fishnet, capturing only nearby messages if they are in the right spot at the right time. Messages are deleted from the server a few minutes after they are sent. This is not only a privacy feature but also a way to keep the server load low. Win-win.</p>
Demo</h2>
In case you want to try a demo, here you go:</p>
catenary.jflessau.de</a></p>
Remember to nag me by opening issues on GitHub if you even whiff a bug or have a feature request: github.com/jflessau/catenary</a></p>


Scheduled Postgres Backups to S3
2022-03-16T00:00:00+00:00
A managed RDS is not exactly cheap. And I like my pet projects to be cheap. That‘s how I ran into the problem of making scheduled backups of a Postgres DB. I can‘t be the only one, so here is one way of solving it with a docker image.</p>
👉 Link to the repository</a></p>
Inside the Docker Container</h2>
Within the running container this happens:</p>

pg_dump</code> creates a dump</li>
aws-cli</code> uploads the dump to s3</li>
cron</code> schedules a job with the two steps above to run every x minutes/houres/etc.</li>
</ul>
Pretty straight forward. But there is a catch.</p>
Version Mismatch</h3>
The pg_dump</code> command may not work across Postgres versions. Meaning you would want the pg_dump</code> command to match the version of the Postgres database you pull a dump from.</p>
Therefore building this image requires the build arg POSTGRES_IMAGE_TAG</code>.

Here is an example of how building the image could look like for Postgres v13.5:</p>
docker</span> build</span> --build-arg </span>"POSTGRES_IMAGE_TAG=13.5"</span> -t</span> pg_dump_s3 .
</span></code></pre>
Internally it pulls the official Postgres image matching the provided tag. Here is a list of available tags for official Postgres images</a>.</p>
Note that this docker image does not work for all of them. E.g. *-alpine</code> images don‘t work. But there is at least one that does for every major version from v9 to v14. Have a look at this list of tested tags</a>.</p>
See the README</a> for detailed information on how to build the image and run the container.</p>
Feel free to open PRs or issues 🙂</p>


Postgres Transaction Isolation
2021-11-25T00:00:00+00:00
In this post we‘ll have a look at transaction isolation levels in Postgres. But first, let‘s have a short recap about transactions and why they‘re needed.</p>
What is a Transaction?</h2>
A transaction combines multiple read and write steps. If a transaction is started but couldn't be completed, it will be rolled back. A rolled back transaction has no effects on the data in the database. It's an all-or-nothing operation.</p>
Example</h2>
Let's say you have got a table for users of an online game:</p>
owner</th> balance</th></tr></thead>

Lisa</td> 2000</td></tr>
</tbody></table>
Users can spend digital coins on stuff in the game.</p>
You want the balance to be correct at any time. No user should be able to spend more than they have and no one should be billed twice for an item.</p>
When Lisa logs in and buys an item for 1000 coins the sql for it could look like this:</p>
-- check the balance to see if Lisa has enough coins
</span>select</span> balance </span>from</span> accounts </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- in your business logic between the queries: check if that's enough and
</span>-- calculate the new balance (1000)
</span>
</span>-- update Lisa's balance
</span>update</span> accounts </span>set</span> balance </span>= </span>1000 </span>where</span> owner </span>= </span>'Lisa'</span>;
</span></code></pre>
That works totally fine and Lisa's new balance is correct.</p>
But what if Lisa is really clever and opens two sessions, one on a phone and one on her laptop?</p>
In this case, she could click the buy-button for two items almost simultaneously:</p>
</p>
In session A Lisa buys an item for 1000</code> coins, so the new balance would be 1000</code>. In Session B she purchases an item for 1250</code> coins, so the new balance would be 750</code>.</p>
The problem is that both sessions read Lisa's initial balance (2000</code> coins) and therefore allow the respective purchases. The latest update query sets her balance to 750</code> coins when she really should have -250</code>. She effectively stole 1000</code> coins 🙀</p>
Transactions can be used to avoid this total nightmare. Wrap the queries of a purchase in a transaction and Postgres ensures that the balances are correct, even when purchases happen concurrently:</p>
-- begin transaction
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- check the balance to see if Lisa has enough coins
</span>select</span> balance </span>from</span> accounts </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- in your business logic between the queries: check if that's enough and
</span>-- calculate the new balance (1000)
</span>
</span>-- update Lisa's balance
</span>update</span> accounts </span>set</span> balance </span>= </span>1000 </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- commit the transaction
</span>commit</span>;
</span></code></pre>
Let's see what happens when we replay the scenario of two concurrent purchases. To simulate this situation we create and populate the table accounts</code>:</p>
create table </span>accounts</span> (owner </span>text primary key</span>, balance </span>integer </span>not </span>null</span>);
</span>
</span>insert into</span> accounts </span>values</span> (</span>'Lisa'</span>, </span>2000</span>);
</span></code></pre>
From now on we need two terminals to simulate Lisa's two user sessions, terminal A and terminal B:</p>
-- TERMINAL A
</span>
</span>-- begin transaction
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- check the balance to see if Lisa has enough coins
</span>select</span> balance </span>from</span> accounts </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- update Lisa's balance
</span>update</span> accounts </span>set</span> balance </span>= </span>1000 </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span></code></pre>
Note that we did not commit the transaction yet, because we want to see what happens when we got two transactions running at the same time. So here comes the second one in terminal B:</p>
-- TERMINAL B
</span>
</span>-- begin transaction
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- check the balance to see if Lisa has enough coins
</span>select</span> balance </span>from</span> accounts </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- update Lisa's balance
</span>update</span> accounts </span>set</span> balance </span>= </span>750 </span>where</span> owner </span>= </span>'Lisa'</span>;
</span></code></pre>
Lisa starts with a balance of 2000</code> coins. In Terminal A we're processing a purchase of an item worth 1000</code> coins, so we're setting the new balance to 1000</code> coins. In terminal B we are processing a purchase worth 1250</code> coins and setting the balance to 750</code>.</p>
Without wrapping the respective queries in transactions this could actually work; Lisa would get both items and a final balance of either 1000</code> or 750</code>, depending on which update query is faster.</p>
Now we commit the transaction in terminal A:</p>
-- TERMINAL A
</span>
</span>-- commit the transaction
</span>commit</span>;
</span></code></pre>
That should work just fine. But now we see this in terminal B:</p>
-- TERMINAL B
</span>
</span>ERROR:  could not serialize access due to concurrent </span>update
</span></code></pre>
and if we try to commit the transaction in terminal B with commit;</code> we get</p>
-- TERMINAL B
</span>
</span>ROLLBACK
</span></code></pre>
Postgres throws an error because it wasn't safe to update the same row that was updated in the transaction of terminal A.</p>
Only the transaction in terminal A was successful and Lisa's new balance is now 1000</code>. Cool! Postgres saved us and Lisa's trick didn't work this time.</p>
Three Isolation Levels of Postgres</h2>
You probably noticed that we initiated the transactions in the example with the transaction isolation level repeatable read</code>:</p>
-- begin transaction
</span>begin</span> transaction isolation level repeatable read;
</span></code></pre>
Postgres has three transaction isolation levels:</p>

read committed</code></li>
repeatable read</code></li>
serializable</code></li>
</ul>
Read committed</code> is the default option in Postgres and it would NOT throw an error in our example. That's why we choose repeatable read</code> and that's why I think it's a good idea check what those isolation levels actually do for us.</p>
Four Things to Worry About</h2>
There are four things to worry about in concurrent transactions:</p>

Dirty read</li>
Repeatable read</li>
Phantom read</li>
Serialization anomaly</li>
</ul>
It's kind of a mismatch: Three isolation levels but four things to worry about. Other database systems have one more isolation level: read uncommitted</code>.</p>
You can actually set the isolation level of a transaction to read uncommited</code>, but internally there is no difference between read committed</code> and read uncommited</code> in Postgres.</p>
Here is what the isolation levels in Postgres allow and what they prevent:</p>
Isolation Level</th> Dirty Read</th> Non repeatable Read</th> Phantom Read</th> Serialization Anomaly</th></tr></thead>

Read uncommitted</td> 🚫</td> ✅</td> ✅</td> ✅</td></tr>
Read committed (default)</td> 🚫</td> ✅</td> ✅</td> ✅</td></tr>
Repeatable read</td> 🚫</td> 🚫</td> 🚫</td> ✅</td></tr>
Serializable</td> 🚫</td> 🚫</td> 🚫</td> 🚫</td></tr>
</tbody></table>
</div>
As you can see, read uncommited</code> does the exact same things as read commited</code> so we'll ignore read uncommited</code> from now on.</p>
To understand what the different isolation levels do, we have to understand the four things to worry about in transactions.</p>
Dirty Read</h3>
A dirty read is a read within a transaction that reads data from another, uncommitted transaction. Dirty reads are never allowed in Postgres transactions, regardless of the isolation level. If you try to create a dirty read within a transaction, this happens:</p>
</p>
You would expect that the read in transaction T2 gets a balance of 1000, but that would be a dirty read, because transaction T1 didn't commit yet.</p>
Non Repeatable Read</h3>
Non repeatable reads are the kind of phenomenon we tried to avoid in the example at the beginning of this article. We used the transaction level repeatable read</code> to ensure that Lisa's balance is correct, even when purchases are happening concurrently.</p>
In the example we had two concurrent transactions that looked like this:</p>
-- begin transaction
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- check the balance to see if Lisa has enough coins
</span>select</span> balance </span>from</span> accounts </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- update Lisa's balance
</span>update</span> accounts </span>set</span> balance </span>= </span>1000 </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- commit the transaction
</span>commit</span>;
</span></code></pre>
When running two of these transactions concurrently with the isolation level repeatable read</code> you'll get an error in transaction T2 once you commit T1. And when you then try to commit T2, T2 will perform a rollback</code>:</p>
</p>
And even if you leave out the select statements...</p>
-- begin transaction
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- update Lisa's balance
</span>update</span> accounts </span>set</span> balance </span>= </span>500 </span>where</span> owner </span>= </span>'Lisa'</span>;
</span>
</span>-- commit the transaction
</span>commit</span>;
</span></code></pre>
... you'll get the same error (could not serialize access due to concurrent update</code>) when trying to run two of these transactions concurrently.</p>
But without the select statements, are we still doing a non repeatable read</strong>? Yes! According to the Postgres docs</a>:</p>

UPDATE</code>, DELETE</code>, SELECT FOR UPDATE</code>, and SELECT FOR SHARE</code> commands behave the same as SELECT</code> in terms of searching for target rows [...]</p>
</blockquote>
Phantom Read</h3>
A phantom read occurs when transaction T1 reads some rows from a table, then another, concurrent transaction T2 adds rows that would be in the selection of T1 and commits, finally T1 performs the same select again, but gets more rows returned because of T2's insertion.</p>
It's super complicated as a sentence, but much simpler in a picture:</p>
</p>
Let's try that in our terminal:</p>
-- (re-) create the accounts table from the first example
</span>create table </span>if not exists </span>accounts</span> (owner </span>text primary key</span>, balance </span>integer </span>not </span>null</span>);
</span>
</span>-- wipe the table
</span>delete from</span> accounts;
</span>
</span>-- insert the first user
</span>insert into</span> accounts </span>values</span> (</span>'Lisa'</span>, </span>2000</span>);
</span></code></pre>
From now on we're again working with two separate terminals (A & B) so that we can create concurrent transactions.</p>
-- TERMINAL A
</span>
</span>-- start transaction that prevents phantom reads
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- select rows from the accounts table
</span>select </span>* </span>from</span> accounts </span>where</span> balance </span>> </span>500</span>;
</span>
</span>-- it returns:
</span>owner | balance
</span>-------+---------
</span> Lisa  |    </span>2000
</span>(</span>1</span> row)
</span></code></pre>
Now we switch to terminal B, start a transaction, insert a new row and commit:</p>
-- TERMINAL B
</span>
</span>-- start transaction
</span>begin</span> transaction isolation level repeatable read;
</span>
</span>-- insert a new row
</span>insert into</span> accounts </span>values</span> (</span>'John'</span>, </span>1250</span>);
</span>
</span>-- commit the transaction
</span>commit</span>;
</span></code></pre>
Now back to terminal A and let's see what the same select statement returns:</p>
-- TERMINAL A
</span>
</span>-- select rows from the accounts table
</span>select </span>* </span>from</span> accounts </span>where</span> balance </span>> </span>500</span>;
</span>
</span>-- it returns:
</span>owner | balance
</span>-------+---------
</span> Lisa  |    </span>2000
</span>(</span>1</span> row)
</span></code></pre>
Ha! We got just Lisa's row. If the select query also returned John's row, that would be a phantom read. But the isolation level we chose (repeatable read</code>) does not allow that, so only Lisa's row is returned.</p>
If you did the same steps with the lower isolation level read committed</code> you would also get John's row, because read committed</code> allows phantom reads:</p>
</p>
Serialization Anomaly</h3>
Serializable is the most restrictive isolation level in Postgres. When using this level, Postgres will prevent serialization anomaly. Meaning it ensures that concurrent transactions are only allowed if the result is the same, regardless of the order in which transactions are processed.</p>
Let's use the accounts</code> table one last time to see how that looks in sql. First we prepare our setup:</p>
-- (re-) create the accounts table from the first example
</span>create table </span>if not exists </span>accounts</span> (owner </span>text primary key</span>, balance </span>integer </span>not </span>null</span>);
</span>
</span>-- wipe the table
</span>delete from</span> accounts;
</span>
</span>-- insert the first user
</span>insert into</span> accounts </span>values</span> (</span>'Lisa'</span>, </span>2000</span>);
</span></code></pre>
Now we want to perform two transactions that produce different results, depending on the order of their execution. Both transactions will try to insert a new row into the table. The balance value will be equal to the sum of all balances in the table.</p>
Again, we're working with two terminals (A & B) to be able to start two concurrent transactions:</p>
-- TERMINAL A
</span>
</span>-- start transaction
</span>begin</span> transaction isolation level serializable;
</span>
</span>-- select rows from the accounts table
</span>insert into</span> accounts </span>select </span>'transaction T1'</span>, </span>sum</span>(balance) </span>from</span> accounts;
</span></code></pre>
-- TERMINAL B
</span>
</span>-- start transaction
</span>begin</span> transaction isolation level serializable;
</span>
</span>-- insert a new row
</span>insert into</span> accounts </span>select </span>'transaction T2'</span>, </span>sum</span>(balance) </span>from</span> accounts;
</span></code></pre>
Depending on which transaction is faster, we expect one of these results:</p>
-- if T1 commits before T2
</span>
</span>select </span>* </span>from</span> accounts;
</span>
</span>owner             | balance
</span>------------------+---------
</span> Lisa             |    </span>2000
</span> transaction T1   |    </span>2000
</span> transaction T2   |    </span>4000
</span></code></pre>
-- if T2 commits before T1
</span>
</span>select </span>* </span>from</span> accounts;
</span>
</span>owner             | balance
</span>------------------+---------
</span> Lisa             |    </span>2000
</span> transaction T1   |    </span>4000
</span> transaction T2   |    </span>2000
</span></code></pre>
If T1 commits before T2, the sum for T1 should be 2000</code>. By the time T2 commits, T2 should get 4000</code> as the sum of all balances (2000</code> for each Lisa</code> and transaction T1</code>).</p>
But the transaction level serializable</code> does not allow both transactions to commit. The first transaction that commits will be fine, but for the second one you commit you will get this:</p>
ERROR:  could not serialize access due to read</span>/</span>write dependencies among transactions
</span>DETAIL:  Reason code: Canceled on identification </span>as</span> a pivot, during </span>commit</span> attempt.
</span>HINT:  The transaction might succeed if retried.
</span></code></pre>
Here is a timeline of the transactions:</p>
</p>
If you did the same transactions with one of the other two isolation levels, both transactions would be able to commit and the result would be:</p>
select </span>* </span>from</span> accounts;
</span>
</span>owner           | balance
</span>----------------+---------
</span> Lisa           |    </span>2000
</span> transaction T1 |    </span>2000
</span> transaction T2 |    </span>2000
</span></code></pre>
And this is not what we expected. Either the balance of transaction T1</code> or transaction T1</code> should be 4000</code>.</p>
Because the order of execution matters for the outcome of these two transactions, the isolation level serializable</code> will only allow one of them to commit.</p>
Postgres again prevented us from ending up with the wrong balance.</p>
We can now retry the failed transaction and given that no other interfering transaction is running, it will commit just fine 🚀</p>
Conclusion</h2>
Transaction isolation is a powerful concept. It helps you to avoid inconsistent data without using a lot more business logic code.</p>


Searchable PDFs
2021-10-13T00:00:00+00:00
Welcome to my first post on this new blog!

In this article I'll explain how I make my ever growing pile of PDFs searchable.</p>
Motivation</h2>
Recently I ran into some situations where I needed to find a specific document. It drove me nuts to manually open and close PDFs to find the one I was looking for.</p>
So I made all my PDFs searchable and here is how that worked:</p>
TL;DR</h2>
Throw all your PDF files into a folder, run ocrmypdf</a> to make them searchable:</p>
find</span> ./</span>*</span>.pdf</span> -type</span> f</span> -name </span>'*.pdf'</span> -not -name </span>'*_ocr.pdf'</span> -exec</span> sh</span> -c </span>'ocrmypdf "${0%.*}.pdf" "${0%.*}_ocr.pdf"' </span>{} </span>\;
</span></code></pre>
then use pdfgrep</a> to search for things:</p>
pdfgrep</span> -i --cache </span>'taxpayer identification number' </span>*</span>.pdf
</span></code></pre>
and never waste time on tagging, renaming or categorizing PDFs again.</p>
Using tesseract & pdfgrep</h2>
There are some really powerful tools out there to get us out of this dilemma. And it's super simple:</p>

Put all PDFs in one place</li>
Let tesseract take care of OCR</li>
Use pdfgrep for search</li>
</ol>
The first step is my favorite. No manual categorizing, no file tags, no problems!</p>
Tesseract</h3>
Tesseract is an open source tool for optical character recognition (OCR). It takes an image and finds text in it:</p>
tesseract</span> input.png output</span> -l</span> eng
</span></code></pre>
Cool! Now we have a way to read text from images. However, this post is all about PDFs. So how can we feed a PDF to tesseract?</p>
Unfortunately, tesseract only works with image formats like PNG, JPEG and some others. You could convert your PDFs to images like this:</p>
convert</span> -density</span> 300 input.pdf</span> -depth</span> 8 output.png
</span></code></pre>
and then use tesseract on the generated images. But there is a simpler way to do this: OCRmyPDF</a>. This is another CLI that does the conversion for you automatically.</p>
ocrmypdf</span> -l</span> eng input.pdf output.pdf
</span></code></pre>
That command works for one file. To run OCR on all PDF files in a folder, use this one:</p>
find</span> ./</span>*</span>.pdf</span> -type</span> f</span> -name </span>'*.pdf'</span> -not -name </span>'*_ocr.pdf'</span> -exec</span> sh</span> -c </span>'ocrmypdf "${0%.*}.pdf" "${0%.*}_ocr.pdf"' </span>{} </span>\;
</span></code></pre>
This will keep the original files and creates copies of those with "_ocr" attached to their filenames. Whenever you add more PDFs to your folder and need to OCR them, just re-run this command and it will try to OCR files without the "_ocr" suffix.</p>
You can either keep the old files or delete all files without the "_ocr" suffix in the current directory by using this command:</p>
find </span>*</span>.pdf !</span> -name </span>'*_ocr.pdf'</span> -delete
</span></code></pre>
Only searchable PDFs are left now. Whenever you throw in more PDFs you will instantly know which need to be treated with ocrmypdf: All those without the _ocr</code> suffix.</p>
pdfgrep</h3>
Now that all your PDFs are searchable, let's try pdfgrep:</p>
pdfgrep</span> -i --cache </span>'invoice' </span>*</span>.pdf
</span></code></pre>
This will give you a list of filenames as well as some characters before/after the matching phrase and the page number.</p>
Note that depending on the amount of PDFs, the first run could take a while. By using the --cache</code> flag the next search will be much quicker!</p>
Using Docker</h2>
If you are familiar with docker, you can also try this docker image</a>. It let's you specify an input and and output directory. You put all your PDFs into the input directory and the running docker container will perform OCR on them and outputs the searchable PDFs to the specified output directory.</p>
Have a look at the Readme</a> for more instructions on how to use and configure it.</p>


Legal Notice
2021-03-06T00:00:00+00:00
Site Notice</h2>
Information pursuant to Sect. 5 German Telemedia Act (TMG)</h3>
Jan Flessau

Meiendorfer Straße 56b

22145 Hamburg</p>
Contact</h3>
Phone: 0175 5950528

E-mail: jan.jf@me.com</p>
VAT ID</h3>
Sales tax identification number according to Sect. 27 a of the Sales Tax Law:
DE323212188</p>
Responsible for the content according to Sect. 55, paragraph 2 RStV</h3>
Jan Flessau

Meiendorfer Straße 56b

22145 Hamburg</p>
EU dispute resolution</h3>
The European Commission provides a platform for online dispute resolution (ODR): https://ec.europa.eu/consumers/odr</a>. Our e-mail address can be found above in the site notice.</p>
Dispute resolution proceedings in front of a consumer arbitration board</h3>
We are not willing or obliged to participate in dispute resolution proceedings in front of a consumer arbitration board.</p>
Liability for Contents</h3>
As service providers, we are liable for own contents of these websites according to Paragraph 7, Sect. 1 German Telemedia Act (TMG). However, according to Paragraphs 8 to 10 German Telemedia Act (TMG), service providers are not obligated to permanently monitor submitted or stored information or to search for evidences that indicate illegal activities. Legal obligations to removing information or to blocking the use of information remain unchallenged. In this case, liability is only possible at the time of knowledge about a specific violation of law. Illegal contents will be removed immediately at the time we get knowledge of them.</p>
Liability for Links</h3>
Our offer includes links to external third party websites. We have no influence on the contents of those websites, therefore we cannot guarantee for those contents. Providers or administrators of linked websites are always responsible for their own contents. The linked websites had been checked for possible violations of law at the time of the establishment of the link. Illegal contents were not detected at the time of the linking. A permanent monitoring of the contents of linked websites cannot be imposed without reasonable indications that there has been a violation of law. Illegal links will be removed immediately at the time we get knowledge of them.</p>
Copyright</h3>
Contents and compilations published on these websites by the providers are subject to German copyright laws. Reproduction, editing, distribution as well as the use of any kind outside the scope of the copyright law require a written permission of the author or originator. Downloads and copies of these websites are permitted for private use only. The commercial use of our contents without permission of the originator is prohibited. Copyright laws of third parties are respected as long as the contents on these websites do not originate from the provider. Contributions of third parties on this site are indicated as such. However, if you notice any violations of copyright law, please inform us. Such contents will be removed immediately.</p>


Privacy Policy
2021-03-06T00:00:00+00:00
1. An overview of data protection</h2>
General information</h3>
The following information will provide you with an easy to navigate overview of what will happen with your personal data when you visit this website. The term “personal data” comprises all data that can be used to personally identify you. For detailed information about the subject matter of data protection, please consult our Data Protection Declaration, which we have included beneath this copy.</p>
Data recording on this website</h3>
Who is the responsible party for the recording of data on this website (i.e. the “controller”)?</h4>
The data on this website is processed by the operator of the website, whose contact information is available under section “Information Required by Law” on this website.</p>
How do we record your data?</h4>
We collect your data as a result of your sharing of your data with us. This may, for instance be information you enter into our contact form.</p>
Other data shall be recorded by our IT systems automatically or after you consent to its recording during your website visit. This data comprises primarily technical information (e.g. web browser, operating system or time the site was accessed). This information is recorded automatically when you access this website.</p>
What are the purposes we use your data for?</h4>
A portion of the information is generated to guarantee the error free provision of the website. Other data may be used to analyze your user patterns.</p>
What rights do you have as far as your information is concerned?</h4>
You have the right to receive information about the source, recipients and purposes of your archived personal data at any time without having to pay a fee for such disclosures. You also have the right to demand that your data are rectified or eradicated. If you have consented to data processing, you have the option to revoke this consent at any time, which shall affect all future data processing. Moreover, you have the right to demand that the processing of your data be restricted under certain circumstances. Furthermore, you have the right to log a complaint with the competent supervising agency.</p>
Please do not hesitate to contact us at any time under the address disclosed in section “Information Required by Law” on this website if you have questions about this or any other data protection related issues.</p>
Analysis tools and tools provided by third parties</h3>
There is a possibility that your browsing patterns will be statistically analyzed when your visit this website. Such analyses are performed primarily with what we refer to as analysis programs.</p>
For detailed information about these analysis programs please consult our Data Protection Declaration below.</p>
2. Hosting and Content Delivery Networks (CDN)</h2>
External Hosting</h3>
This website is hosted by an external service provider (host). Personal data collected on this website are stored on the servers of the host. These may include, but are not limited to, IP addresses, contact requests, metadata and communications, contract information, contact information, names, web page access, and other data generated through a web site.
The host is used for the purpose of fulfilling the contract with our potential and existing customers (Art. 6 para. 1 lit. b GDPR) and in the interest of secure, fast and efficient provision of our online services by a professional provider (Art. 6 para. 1 lit. f GDPR).
Our host will only process your data to the extent necessary to fulfil its performance obligations and to follow our instructions with respect to such data.</p>
Execution of a contract data processing agreement</h4>
In order to guarantee processing in compliance with data protection regulations, we have concluded an order processing contract with our host.</p>
3. General information and mandatory information</h2>
Data protection</h3>
The operators of this website and its pages take the protection of your personal data very seriously. Hence, we handle your personal data as confidential information and in compliance with the statutory data protection regulations and this Data Protection Declaration.</p>
Whenever you use this website, a variety of personal information will be collected. Personal data comprises data that can be used to personally identify you. This Data Protection Declaration explains which data we collect as well as the purposes we use this data for. It also explains how, and for which purpose the information is collected.</p>
We herewith advise you that the transmission of data via the Internet (i.e. through e-mail communications) may be prone to security gaps. It is not possible to completely protect data against third party access.</p>
Information about the responsible party (referred to as the “controller” in the GDPR)</h3>
The data processing controller on this website is:

Jan Flessau

Meiendorfer Straße 56b

22145 Hamburg

Phone: 0175 5950528

E-mail: jan.jf@me.com</p>
The controller is the natural person or legal entity that single-handedly or jointly with others makes decisions as to the purposes of and resources for the processing of personal data (e.g. names, e-mail addresses, etc.).</p>
Revocation of your consent to the processing of data</h3>
A wide range of data processing transactions are possible only subject to your express consent. You can also revoke at any time any consent you have already given us. This shall be without prejudice to the lawfulness of any data collection that occurred prior to your revocation.</p>
Right to object to the collection of data in special cases; right to object to direct advertising (Art. 21 GDPR)</h3>
IN THE EVENT THAT DATA ARE PROCESSED ON THE BASIS OF ART. 6 SECT. 1 LIT. E OR F GDPR, YOU HAVE THE RIGHT TO AT ANY TIME OBJECT TO THE PROCESSING OF YOUR PERSONAL DATA BASED ON GROUNDS ARISING FROM YOUR UNIQUE SITUATION. THIS ALSO APPLIES TO ANY PROFILING BASED ON THESE PROVISIONS. TO DETERMINE THE LEGAL BASIS, ON WHICH ANY PROCESSING OF DATA IS BASED, PLEASE CONSULT THIS DATA PROTECTION DECLARATION. IF YOU LOG AN OBJECTION, WE WILL NO LONGER PROCESS YOUR AFFECTED PERSONAL DATA, UNLESS WE ARE IN A POSITION TO PRESENT COMPELLING PROTECTION WORTHY GROUNDS FOR THE PROCESSING OF YOUR DATA, THAT OUTWEIGH YOUR INTERESTS, RIGHTS AND FREEDOMS OR IF THE PURPOSE OF THE PROCESSING IS THE CLAIMING, EXERCISING OR DEFENCE OF LEGAL ENTITLEMENTS (OBJECTION PURSUANT TO ART. 21 SECT. 1 GDPR).
IF YOUR PERSONAL DATA IS BEING PROCESSED IN ORDER TO ENGAGE IN DIRECT ADVERTISING, YOU HAVE THE RIGHT TO AT ANY TIME OBJECT TO THE PROCESSING OF YOUR AFFECTED PERSONAL DATA FOR THE PURPOSES OF SUCH ADVERTISING. THIS ALSO APPLIES TO PROFILING TO THE EXTENT THAT IT IS AFFILIATED WITH SUCH DIRECT ADVERTISING. IF YOU OBJECT, YOUR PERSONAL DATA WILL SUBSEQUENTLY NO LONGER BE USED FOR DIRECT ADVERTISING PURPOSES (OBJECTION PURSUANT TO ART. 21 SECT. 2 GDPR).</p>
Right to log a complaint with the competent supervisory agency</h3>
In the event of violations of the GDPR, data subjects are entitled to log a complaint with a supervisory agency, in particular in the member state where they usually maintain their domicile, place of work or at the place where the alleged violation occurred. The right to log a complaint is in effect regardless of any other administrative or court proceedings available as legal recourses.</p>
Right to data portability</h3>
You have the right to demand that we hand over any data we automatically process on the basis of your consent or in order to fulfil a contract be handed over to you or a third party in a commonly used, machine readable format. If you should demand the direct transfer of the data to another controller, this will be done only if it is technically feasible.</p>
SSL and/or TLS encryption</h3>
For security reasons and to protect the transmission of confidential content, such as purchase orders or inquiries you submit to us as the website operator, this website uses either an SSL or a TLS encryption program. You can recognize an encrypted connection by checking whether the address line of the browser switches from “http://” to “https://” and also by the appearance of the lock icon in the browser line.</p>
If the SSL or TLS encryption is activated, data you transmit to us cannot be read by third parties.</p>
Information about, rectification and eradication of data</h3>
Within the scope of the applicable statutory provisions, you have the right to at any time demand information about your archived personal data, their source and recipients as well as the purpose of the processing of your data. You may also have a right to have your data rectified or eradicated. If you have questions about this subject matter or any other questions about personal data, please do not hesitate to contact us at any time at the address provided in section “Information Required by Law.”</p>
Right to demand processing restrictions</h3>
You have the right to demand the imposition of restrictions as far as the processing of your personal data is concerned. To do so, you may contact us at any time at the address provided in section “Information Required by Law.” The right to demand restriction of processing applies in the following cases:</p>

In the event that you should dispute the correctness of your data archived by us, we will usually need some time to verify this claim. During the time that this investigation is ongoing, you have the right to demand that we restrict the processing of your personal data.</li>
If the processing of your personal data was/is conducted in an unlawful manner, you have the option to demand the restriction of the processing of your data in lieu of demanding the eradication of this data.</li>
If we do not need your personal data any longer and you need it to exercise, defend or claim legal entitlements, you have the right to demand the restriction of the processing of your personal data instead of its eradication.</li>
If you have raised an objection pursuant to Art. 21 Sect. 1 GDPR, your rights and our rights will have to be weighed against each other. As long as it has not been determined whose interests prevail, you have the right to demand a restriction of the processing of your personal data.</li>
</ul>
If you have restricted the processing of your personal data, these data – with the exception of their archiving – may be processed only subject to your consent or to claim, exercise or defend legal entitlements or to protect the rights of other natural persons or legal entities or for important public interest reasons cited by the European Union or a member state of the EU.</p>
Rejection of unsolicited e-mails</h3>
We herewith object to the use of contact information published in conjunction with the mandatory information to be provided in section “Information Required by Law” to send us promotional and information material that we have not expressly requested. The operators of this website and its pages reserve the express right to take legal action in the event of the unsolicited sending of promotional information, for instance via SPAM messages.</p>
4. Recording of data on this website</h2>
Cookies</h3>
Our websites and pages use what the industry refers to as “cookies.” Cookies are small text files that do not cause any damage to your device. They are either stored temporarily for the duration of a session (session cookies) or they are permanently archived on your device (permanent cookies). Session cookies are automatically deleted once you terminate your visit. Permanent cookies remain archived on your device until you actively delete them or they are automatically eradicated by your web browser.
In some cases it is possible that third party cookies are stored on your device once you enter our site (third party cookies). These cookies enable you or us to take advantage of certain services offered by the third party (e.g. cookies for the processing of payment services).
Cookies have a variety of functions. Many cookies are technically essential since certain website functions would not work in the absence of the cookies (e.g. the shopping cart function or the display of videos). The purpose of other cookies may be the analysis of user patterns or the display of promotional messages.
Cookies, which are required for the performance of electronic communication transactions (required cookies) or for the provision of certain functions you want to use (functional cookies, e.g. for the shopping cart function) or those that are necessary for the optimization of the website (e.g. cookies that provide measurable insights into the web audience), shall be stored on the basis of Art. 6 Sect. 1 lit. f GDPR, unless a different legal basis is cited. The operator of the website has a legitimate interest in the storage of cookies to ensure the technically error free and optimized provision of the operator’s services. If your consent to the storage of the cookies has been requested, the respective cookies are stored exclusively on the basis of the consent obtained (Art. 6 Sect. 1 lit. a GDPR); this consent may be revoked at any time.</p>
You have the option to set up your browser in such a manner that you will be notified any time cookies are placed and to permit the acceptance of cookies only in specific cases. You may also exclude the acceptance of cookies in certain cases or in general or activate the delete function for the automatic eradication of cookies when the browser closes. If cookies are deactivated, the functions of this website may be limited.
In the event that third party cookies are used or if cookies are used for analytical purposes, we will separately notify you in conjunction with this Data Protection Policy and, if applicable, ask for your consent.</p>
Server log files</h3>
The provider of this website and its pages automatically collects and stores information in so-called server log files, which your browser communicates to us automatically. The information comprises:</p>

The type and version of browser used</li>
The used operating system</li>
Referrer URL</li>
The hostname of the accessing computer</li>
The time of the server inquiry</li>
The IP address</li>
</ul>
This data is not merged with other data sources.</p>
This data is recorded on the basis of Art. 6 Sect. 1 lit. f GDPR. The operator of the website has a legitimate interest in the technically error free depiction and the optimization of the operator’s website. In order to achieve this, server log files must be recorded.</p>
Request by e-mail, telephone or fax</h3>
If you contact us by e-mail, telephone or fax, your request, including all resulting personal data (name, request) will be stored and processed by us for the purpose of processing your request. We do not pass these data on without your consent.
These data are processed on the basis of Art. 6 Sect. 1 lit. b GDPR if your inquiry is related to the fulfillment of a contract or is required for the performance of pre-contractual measures. In all other cases, the data are processed on the basis of our legitimate interest in the effective handling of inquiries submitted to us (Art. 6 Sect. 1 lit. f GDPR) or on the basis of your consent (Art. 6 Sect. 1 lit. a GDPR) if it has been obtained.
The data sent by you to us via contact requests remain with us until you request us to delete, revoke your consent to the storage or the purpose for the data storage lapses (e.g. after completion of your request). Mandatory statutory provisions - in particular statutory retention periods - remain unaffected.</p>
5. Newsletter</h2>
Newsletter data</h3>
If you would like to subscribe to the newsletter offered on this website, we will need from you an e-mail address as well as information that allow us to verify that you are the owner of the e-mail address provided and consent to the receipt of the newsletter. No further data shall be collected or shall be collected only on a voluntary basis. We shall use such data only for the sending of the requested information and shall not share such data with any third parties.</p>
The processing of the information entered into the newsletter subscription form shall occur exclusively on the basis of your consent (Art. 6 Sect. 1 lit. a GDPR). You may revoke the consent you have given to the archiving of data, the e-mail address and the use of this information for the sending of the newsletter at any time, for instance by clicking on the “Unsubscribe” link in the newsletter. This shall be without prejudice to the lawfulness of any data processing transactions that have taken place to date.</p>
The data deposited with us for the purpose of subscribing to the newsletter will be stored by us until you unsubscribe from the newsletter or the newsletter service provider and deleted from the newsletter distribution list after you unsubscribe from the newsletter. Data stored for other purposes with us remain unaffected.</p>
After you unsubscribe from the newsletter distribution list, your e-mail address may be stored by us or the newsletter service provider in a blacklist to prevent future mailings. The data from the blacklist is used only for this purpose and not merged with other data. This serves both your interest and our interest in complying with the legal requirements when sending newsletters (legitimate interest within the meaning of Art. 6 para. 1 lit. f GDPR). The storage in the blacklist is indefinite. You may object to the storage if your interests outweigh our legitimate interest.</p>
Jan's Blog

Retro Cube

Features</h3> The Retro Cube has three views. You can switch between them by rotating the dial. If you press it down, the display turns off completely, which can be useful at night. The three views are:</p> Clock</li> Weather dashboard</li>

Gasoline Price Cycles

sqlx-fmt

Usage</h2> This CLI has two commands: format</code> and check</code>.</p>

Retweet 2 Win

Ritmo

Sitcom as a Service

Coding with AI

InkDay

Back</h3> </p>

Website Screenshot</h3> </p>

Beam Base

Individual Parts</h3> </p>

Narrow Beam</h3> </p>

Wide Beam</h3> </p>

Postgres Ultras T-Shirt

SurrealDB

Test: Query Table with 1 Mio. Records</h3> I populated a user</code> table with one million rows in both SurrealDB and Postgres for some query benchmarks. I'm using SurrealDB 2.1.2 with the in-memory KV store and Postgres 17.0, both running in Docker on an M1 Mac.</p>

Local vs. Cloud</h3> And for good measure, I did the exact same measurements on a DB instance in SurrealDB's cloud. Just in case my way of running the DB locally was the bottleneck. Here is a summary of all results</p>

Drawer Inserts

BS Bingo

Catenary

Solution</h2> After many months since that train trip, I'm finally bringing this idea to life. Here's how it works:</p>

Scheduled Postgres Backups to S3

Inside the Docker Container</h2> Within the running container this happens:</p> pg_dump</code> creates a dump</li> aws-cli</code> uploads the dump to s3</li>

Postgres Transaction Isolation

What is a Transaction?</h2> A transaction combines multiple read and write steps. If a transaction is started but couldn't be completed, it will be rolled back. A rolled back transaction has no effects on the data in the database. It's an all-or-nothing operation.</p>

Searchable PDFs

Motivation</h2> Recently I ran into some situations where I needed to find a specific document. It drove me nuts to manually open and close PDFs to find the one I was looking for.</p> So I made all my PDFs searchable and here is how that worked:</p>

Using tesseract & pdfgrep</h2> There are some really powerful tools out there to get us out of this dilemma. And it's super simple:</p>

Tesseract</h3> Tesseract is an open source tool for optical character recognition (OCR). It takes an image and finds text in it:</p>

Legal Notice

Site Notice</h2>

Information pursuant to Sect. 5 German Telemedia Act (TMG)</h3> Jan Flessau Meiendorfer Straße 56b 22145 Hamburg</p>

Contact</h3> Phone: 0175 5950528 E-mail: jan.jf@me.com</p>

VAT ID</h3> Sales tax identification number according to Sect. 27 a of the Sales Tax Law: DE323212188</p>

Responsible for the content according to Sect. 55, paragraph 2 RStV</h3> Jan Flessau Meiendorfer Straße 56b 22145 Hamburg</p>

Privacy Policy

1. An overview of data protection</h2>

Data recording on this website</h3>

Who is the responsible party for the recording of data on this website (i.e. the “controller”)?</h4> The data on this website is processed by the operator of the website, whose contact information is available under section “Information Required by Law” on this website.</p>

What are the purposes we use your data for?</h4> A portion of the information is generated to guarantee the error free provision of the website. Other data may be used to analyze your user patterns.</p>

2. Hosting and Content Delivery Networks (CDN)</h2>

Execution of a contract data processing agreement</h4> In order to guarantee processing in compliance with data protection regulations, we have concluded an order processing contract with our host.</p>

3. General information and mandatory information</h2>

4. Recording of data on this website</h2>

5. Newsletter</h2>

Back</h3>
</p>

Website Screenshot</h3>
</p>

Individual Parts</h3>
</p>

Narrow Beam</h3>
</p>

Wide Beam</h3>
</p>

Test: Query Table with 1 Mio. Records</h3>
I populated a `user</code> table with one million rows in both SurrealDB and Postgres for some query benchmarks. I'm using SurrealDB 2.1.2 with the in-memory KV store and Postgres 17.0, both running in Docker on an M1 Mac.</p>`

Local vs. Cloud</h3>
And for good measure, I did the exact same measurements on a DB instance in SurrealDB's cloud. Just in case my way of running the DB locally was the bottleneck. Here is a summary of all results</p>

What is a Transaction?</h2>
A transaction combines multiple read and write steps. If a transaction is started but couldn't be completed, it will be rolled back. A rolled back transaction has no effects on the data in the database. It's an all-or-nothing operation.</p>

Motivation</h2>
Recently I ran into some situations where I needed to find a specific document. It drove me nuts to manually open and close PDFs to find the one I was looking for.</p>
So I made all my PDFs searchable and here is how that worked:</p>

Information pursuant to Sect. 5 German Telemedia Act (TMG)</h3>
Jan Flessau
Meiendorfer Straße 56b
22145 Hamburg</p>

Contact</h3>
Phone: 0175 5950528
E-mail: jan.jf@me.com</p>

VAT ID</h3>
Sales tax identification number according to Sect. 27 a of the Sales Tax Law: DE323212188</p>

Responsible for the content according to Sect. 55, paragraph 2 RStV</h3>
Jan Flessau
Meiendorfer Straße 56b
22145 Hamburg</p>

Who is the responsible party for the recording of data on this website (i.e. the “controller”)?</h4>
The data on this website is processed by the operator of the website, whose contact information is available under section “Information Required by Law” on this website.</p>

What are the purposes we use your data for?</h4>
A portion of the information is generated to guarantee the error free provision of the website. Other data may be used to analyze your user patterns.</p>

Execution of a contract data processing agreement</h4>
In order to guarantee processing in compliance with data protection regulations, we have concluded an order processing contract with our host.</p>
