Posts on Mitja Martini

Senior means simpler

Tue, 27 Jan 2026 00:00:00 +0000

When I was young, I thought of senior engineers, consultants, etc. as experts that are more effective. My assumption was that seniors work harder to get there and be senior.

Now that I’m older, I realize: Being efficient is as important and probably more distintctive than being effective: I need to be calm, save time, and work efficiently to achieve the goals in a relaxed way, as I don’t have as much energy anymore and because time feels much more precious.

Being senior is all about simplifying.

Export ChatGPT Conversations as Markdown

Fri, 23 Jan 2026 00:00:00 +0000

Today I learned about Rashid’s solution to export ChatGPT conversations to Markdown or even PDF if you like. I prefer Markdown as I can use it to continue the conversation with Claude for example, or change the content and use it in other contexts.

You don’t need to install anything because it’s just a JavaScript snippet. You open the conversation in a browser, select anything in the conversation, inspect it with developer tools and use the console to paste and run the JavaScript snippet and then the Markdown gets downloaded. It’s really simple and extremely helpful in my view.

Here’s the link: Export Your ChatGPT Conversations to Markdown & PDF

Hardware for local coding models is still affordable. For how long?

Sat, 17 Jan 2026 00:00:00 +0000

The recent RAM price hikes have pushed GPU prices up as well. The only systems that have not yet been affected to the same extent are Macs and high-end GPUs (RTX 6000 Pro and above). However, I would already classify GPUs as out of reach: running coding models with sufficiently large context windows would require one or two RTX 6000 Pro cards, or three to six RTX 5090s.

Let’s take a look at Macs instead.

Price	Mem Bandwidth	CPU	RAM	NVMe	CPU	GPU	Model	Price/Bandwidth	Price/Memory
2.500 EUR	273 GB/s	M4 Pro	64 GB	1 TB	12C	16C	Mac Mini	9,15 EUR/GB/s	39,06 EUR/GB
4.027 EUR	546 GB/s	M4 Max	128 GB	1 TB	16C	40C	Mac Studio	7,14 EUR/GB/s	31,43 EUR/GB
4.200 EUR	800 GB/s	M3 Ultra	96 GB	1 TB	28C	60C	Mac Studio	5,52 EUR/GB/s	43,75 EUR/GB
6.720 EUR	800 GB/s	M3 Ultra	256 GB	2 TB	28C	60C	Mac Studio	8,40 EUR/GB/s	26,25 EUR/GB
11.900 EUR	800 GB/s	M3 Ultra	512 GB	4 TB	32C	80C	Mac Studio	14,87 EUR/GB/s	23,24 EUR/GB

Mac Studios with the M3 Ultra currently offer the best overall value. The 96 GB version is fast, and its RAM capacity is sufficient to run many general-purpose models using quantization. The 256 GB version can handle coding models at reasonable quantization levels with 64k context, which is just about sufficient for running coding agents ¹. The 512 GB model, however, exceeds the €10k, which I consider a hard upper limit for what can reasonably be called an “affordable” system.

The price difference between the 96 GB and 256 GB Mac Studio models is surprisingly close to current RAM market prices. For comparison, 256 GB of DDR5-6000 RAM currently costs around €3,430, or €13.39 per GB. The price delta between the 96 GB and 256 GB Mac Studio, and between the 256 GB and 512 GB versions, is approximately €12.88 per GB and €18.50 per GB, respectively. If Apple were to return to its usual memory pricing premiums, the 256 GB configuration would likely end up much closer to €10,000.

Hosted inference is both cheaper and more capable, but it cannot be used in all scenarios. In particular, freelancers and small companies are often required to rely on local systems when working for larger clients. The M3 Ultra, launched in March 2025, together with improved models, quantization techniques, and REAP, has effectively kicked off an era of affordable local coding models. That era may now enter a pause: RAM prices are expected to rise through 2026 and are likely to remain elevated until 2027 or 2028 ².

Based on benchmark results shared on r/localllama ↩︎
Based on an analysis from December 2025 on wccftech ↩︎

MCP and A2A Attack Vectors for AI Agents

Wed, 14 Jan 2026 00:00:00 +0000

Christian Posta from Solo.io has written an interesting Deep Dive into MCP and A2A Attack Vectors for AI Agents.

Here is a shortlist. There are certainly more attack vectors, and the mitigations are a start, but certainly not perfect.

Attack Vector	How it works	Mitigation
Naming Attacks	Attackers create look-alike names or typosquatted services that trick the AI into picking a malicious resource instead of the legitimate one	Enforce unique identities, cryptographically verify server/agent identities, and use a trusted registry rather than blind name matching
Context Poisoning / Indirect Prompt Injection	An attacker embeds hidden instructions in context that steer the model to do harmful actions or leak data; in A2A this can happen via malicious task states or malformed skill descriptions	Sanitize and vet descriptions, use strict schema constraints, and limit or filter natural-language metadata that models see
Shadowing Attacks	A malicious service shadows a legitimate tool or agent by registering something that alters how other trusted components behave (e.g., injecting hidden guidance that changes billing logic or influences other agents’ outputs)	Require authentication and authorization for every component, use whitelists, and ensure that tools/agents don’t get used solely based on unverified contextual presence
Rug Pulls	An attacker initially builds trust by providing a useful tool or agent, but once widely adopted, subtly changes its behavior to perform harmful operations, manipulate outputs, or exfiltrate data	Continuous monitoring, evaluate behavioral changes over time, use policy controls and version gating so tools can’t suddenly change semantics without review

AI agents use protocols like MCP (Model Context Protocol) and A2A (Agent-to-Agent) and decide when and how to call tools or other agents. They do this dynamic discovery and usage based on natural language context which makes them susceptible to semantic manipulation.

Try vibe coding (again)

Tue, 13 Jan 2026 20:39:52 +0100

In case you haven’t tried vibe coding, recently, you should probably try it (again). Their performance has increased quite a bit thanks to recent models like Opus 4.5. Here are some recent takes on vibe coding:

Everyone’s talking about vibe coding without looking at code. I was skeptical. I decided to give it a shot on a challenging problem and was blown away by what I could accomplish in 8 hours. I’m not skeptical anymore, but I also do NOT think it kills SaaS.

– Hamel Husain on Youtube

I’m not joking and this isn’t funny. We have been trying to build distributed agent orchestrators at Google since last year. There are various options, not everyone is aligned… I gave Claude Code a description of the problem, it generated what we built last year in an hour.

– Jaana Dogan, Principal Engineer at Google on X

I’ve never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There’s a new programmable layer of abstraction to master […] Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession.

Roll up your sleeves to not fall behind.

– Andrej Karpathy on X

The Van Halen Test can check if an agent knows it's context

Tue, 13 Jan 2026 19:43:45 +0100

Here’s a trick from the 80s that’s still useful, today: Van Halen’s live shows were potentially dangerous. To make sure the local crew read the rooster, they asked for a candy bowl with M&Ms and added:

Warning: Absolutely no brown ones.

The Van Halen Test via Left hand writes

One day in 1982, they trashed their backstage area because David Lee Roth found a brown M&M.

With coding agents, it’s also not always clear, if they still know all the context. Adding an easy-to-check piece of information helps finding out. I hope that won’t make you trash your office, though.

From 8 Lines with Dokku to 200 with Kubernetes – Why I'm Still Switching

Tue, 13 Jan 2026 18:21:46 +0200

So far, my web apps run on a Dokku server. I haven’t tried Vercel or Fly because I didn’t want to deal with complex pricing models that incur more costs with every additional project.

Dokku works like Heroku:

dokku apps:create myapp
dokku postgres:create myapp-db
dokku postgres:link myapp-db myapp
dokku config:set myapp APP_SECRET="$(openssl rand -base64 48)"
dokku git:from-image myapp myregistry.example.com/image:tag
dokku domains:set myapp myapp.mitjas.com
dokku ports:set myapp http:80:8000 https:443:8000
dokku letsencrypt:enable myapp

It doesn’t get any simpler. After just 8 lines I have

an app on the server,
a PostgreSQL DB with a user and password,
the app linked to Postgres,
a secret configured,
the app deployed from a Docker image,
a reverse proxy configured to forward HTTP and HTTPS to the app’s container, and
TLS with certificates signed by Let’s Encrypt (including automatic renewal).

With Kubernetes I need almost 200 lines of YAML for this. Why do I still want to switch to Kubernetes?

Dokku is great for small projects that can make do with the Dokku plugins and run on a single server. But I believe Kubernetes is a better fit for me in the long run.

Kubernetes manifests tell me (and LLMs) what’s running on the cluster, and I can continuously develop and improve them. Dokku, in contrast, feels like “fire, forget, and start from scratch.” Other advantages of Kubernetes like better scalability, more choice, and security are nice, too, but for me the most important advantage right now is:

Kubernetes is well-suited for Vibe Coding.

That’s my assumption, anyway. Let’s see how it goes.

What Takes Time in Vibe Coding

Tue, 13 Jan 2026 00:00:00 +0000

Vibe scripting, for me, is when I develop small tools for myself with the help of coding agents. It works extremely well, especially for command-line tools.

I recently told a tax advisor about it. He was interested and wanted to see how it works. So I developed a small example on the spot: a VAT calculator. Not the best example, but I couldn’t think of anything better on short notice.

It worked well, and after five minutes the VAT calculator with a Flet/Flutter GUI was up and running.

But I could also see: there’s still room for improvement. For example, the layout wasn’t great and the functionality was too limited.

As i wanted to know how long it would take to turn it into an actually useful app, I later developed it to become a VAT calculator for all EU countries, which uses a small AI pipeline to load rates and descriptions of which product categories are subject to which VAT rate from official EU pages and displays them in an improved GUI.

The EU VAT Calculator after 2h dev time

What started with five minutes for the simple version turned into about two hours. Of course, it’s much better now. But it’s interesting how big the effort difference is between a solution that serves a specific, narrowly defined purpose and a tool with a broader scope. There’s a lot of work involved, and finesse is needed to make a tool that’s truly useful. Personally, I need iterations with a human in the loop for that. Maybe there are developers who can perfectly specify everything upfront, but I usually need to see and use something to evaluate and improve it.

AI accelerates iterations enormously, and you can decide whether to consider it “good enough” sooner or do a few more iterations. I think this is an important reason why I don’t develop faster with AI. I invest the time in more iterations and perhaps less thinking upfront, which then requires more iterations again.

MCP in ChatGPT Developer Mode Beta

Fri, 17 Oct 2025 00:00:00 +0000

OpenAI just releasesd MCP connectors and ChatGPT developer mode beta. In this post, I describe the process of connecting MCP servers to ChatGPT, show how they look and feel right now in a chat session and give an overview of their current limitations.

Activating developer mode
#

MCP connectors can only be created and edited in developer mode which can look a bit scary:

Note: This article is written from the perspective of a Pro account user. OpenAI’s developer mode and MCP connectors in ChatGPT documentation describes that admins can publish MCPs in their organization. I cannot test this but I assume that MCP connectors are then also usable in normal mode.

Developer mode can be activated in Settings > Apps & Connectors > Advanced Settings.

Creating an MCP connection
#

Once the developer mode is active, MCP server connections can be added in Settings > Apps & Connectors:

Connectors take an icon, name, description, the MCP server url, and authentication information. Both SSE and streaming transports are supported and OAuth 2.0 can be used for authentication.

Using an MCP connection
#

MCP connections need to be activated for each chat session:

In my testing, ChatGPT only uses an MCP connection when it has been instructed to do so. Actions need to be confirmed, but decisions can be stored for the duration of a chat:

Note: ChatGPT marks this action as a write action, even though it’s really a read action from a user point of view. I don’t know if it’s a mistake on the side of the DeepWiki MCP, but the interesting part for me is, that Pro accounts seem to support write actions, already, even though this is still documented as a limitation.

MCP connections cannot be added to custom GPTs. Thus, there is no way to preconfigure the custom instructions needed to call an MCP server together with the MCP connection.

Current Limitations
#

Currently, there are still quite a few limitations:

Not available on free accounts.
Pro accounts can only use read/fetch actions (according to the documentation, this might not be true anymore)
local MCPs are not possible.
Agent mode does not support custom connectors.
Deep research mode only supports read/fetch actions.
Only available on the web, not in the mobile app. In the desktop app, connected MCPs are visible but cannot be used (in Pro accounts), as the desktop app does not support developer mode. I assume MCPs might be usable in Business and Enterprise/Edu accounts.

Trying to make sense of it
#

Right now, I think MCP connectors in ChatGPT are not yet ready for day-to-day use - no wonder, it’s a beta, after all. Meanwhile, actions in custom GPTs are still usable, more open and not as scary to use.

OpenAI could have added MCP connectors to custom GPTs and provided an option to pre-confirm actions for certain MCP servers but decided to go a different route with the developer mode.

For me, MCP is a way to customize chatbots and give them “agency”. I find use-case-specific MCP servers better than generic ones which tend to bloat the context and distract the LLM.

I hope, OpenAI will evolve ChatGPT’s MCP connectors without introducing an obligatory validation procedure, but I won’t bet on it, right now.

Claude Code in Devcontainers

Tue, 14 Oct 2025 00:00:00 +0000

Development Containers or just “devcontainers” add a layer of security, simplify developer onboarding enables developing in parallel with isolated environments.

For me, Devcontainers are a great addition to an AI Engineer’s toolbox, even though I don’t use them day-to-day.

I have setup Devcontainers with Claude Code using VS Code and the devcontainers cli, adapted it to a Python FastAPI based project, and documented it in this article.

Tipps for Migrating to Hugo

Sat, 06 Sep 2025 00:00:00 +0000

In this article, I document some non-technical things I’ve learned migrating my blog from Jekyll to Hugo. The gist is to keep it as simple as possible:

Evals for Voice Agents (Session Notes)

Mon, 30 Jun 2025 00:00:00 +0000

Notes of a whirlwind intro to evals for voice agents by Kwindla and swyx

Coding Agents as Slot Machines

Sun, 01 Jun 2025 00:00:00 +0000

After reading a nice post via HN about On Stepping Back, I habitually scrolled through the comments and found this nugget by evrimoztamur:

Interacting with LLM coding tools is much like playing a slot machine, it grabs and chokeholds your gambling instincts. You’re rolling dice for the perfect result without much thought.

This probably applies to many systems that use Generative AI, especially systems that work on the cutting edge where randomness is more pronounced.

If you also “Can’t sleep gud anymore” as Mario Zechner beautifully put it, be aware that coding with agents can be addictive.

Pipecat Cloud Latency for EU Users

Sun, 11 May 2025 00:00:00 +0000

Pipecat Cloud is located in the US. Is its latency ok for voice agents for EU Users.

K/V Cache Quantization in Ollama

Sat, 10 May 2025 00:00:00 +0000

A somewhat hidden feature of Ollama is K/V Cache quantization. This is relevant for local AI as it reduces memory consumption, especially for small LLMs with large context windows.

This post describes how to activate K/V Cache in Ollama and gives an overview of its benefits, drawbacks and use cases.

Activating K/V cache quantization in Ollama
#

K/V Cache quantization in Ollama is not on by default, so you need to activate it by setting the OLLAMA_KV_CACHE_TYPE environment variable. Supported values are documented in How can I set the quantization type for the K/V cache?:

f16
q8_0
q4_0

Benefits and Drawbacks
#

K/V Cache quantization can be the difference between being able to run a model on a machine, or not. You can use Sam McLeod’s vram-estimator to estimate the memory consumption of models with different quantization settings.

The benefit in brief is lower memory consumption which is most pronounced when using small models with large context windows.

The main drawback is reduced model accuracy. The lmdeploy team has written a blog post about K/V cache quantization accuracy test results if you want to see quantified impacts of K/V quantization on model accuracy.

Example numbers
#

Llama 3.2 8B supports 128.000 tokens context windows.

When you run it with Q4_K_M quantization and the longest possible context length, it consumes

23.3 GB memory without K/V cache qantization,
17.0 GB with Q8_K_0 K/V cache quantization, and
13.8 GB with Q4_K_0 K/V cache quantization.

With Q4_K_0 K/V cache quantization it now fits into 16 GB vRAM.

Use cases
#

Application	Benefit
code generation	more code in context
question answering	whole docs fit into the context
function calling	more tools
chat	longer conversations
multi-modal	images need many tokens

More Info
#

To learn more, read Sam McLeod’s in-dept blog post about Bringing K/V Context Quantisation to Ollama. Sam helped to implement this in Ollama.

Deploying Voice Agents to Production

Fri, 09 May 2025 00:00:00 +0000

Here are my notes on the session about deploying voice agents to production which is part of the Voice Agents Course.

The course if held by kwindla und swyx, the CTO und an investor of Daily.co, a WebRTC und Voice AI infrastructure provider. Some of their recommendations might be predisposed. I still state them as is as I trust them and because I don’t have enough experience with voice agents in production.

TL;DR
#

Use a voice AI provider for simple, scalable deployment for production.
Use a single VM or your homelab for demos.

Differences between voice agents and traditional web apps
#

are mostly in the transport
persistent connnection (minutes)
bidirectional streaming
stateful sessions

Voice agents in production need
#

A http service for
- API endpoints,
- a website, and
- webhooks,
- spawning bots.
A media transport layer or service:
- WebRTC based for client-to-server (udp), or
- websocket based for server-to-server (tcp).
Bots (udp or tcp, connect to media transport layer)

Bots
#

Are instances of the agents.
Can be written in Python with PipeCat.
Use STT, LLM, and TTS providers, which also are the main cost and latency drivers.
Usually come packaged with small models, eg. for voice activity detection (VAD)
Each spawned bot serves one session and needs allocated resources during the whole session:
- 0,5 vCPU
- 1 GB RAM
- 40kbps for WebRTC audio (in 30-60 kbps range)
- video requires more CPU (eg. 1 vCPU), and bandwidth
Need to be quickly available. Target time-to-first-word:
- 2-3 secs (web),
- 3-5 secs (phone)

Ways to solve the “fast start challenge”
#

percentage-based warm pool
fast startup times (caching, pre-loading)
proactive/predictive scheduling
fallbacks from reactive world (eg. UX based solutions, not just silent fails)

Infra providers
#

need to support tcp and udp
voice ai providers are easiest (Pipecat Cloud, Daily, Vapi, Layercode)
Fly.io (and potentially other container platforms) are good if they support udp (Fly does)
ML focused provides are good for converged bots with larger models included (gpu clouds)
hyperscalers are flexible but complex
BTW: CloudRun does not support udp
demos can run on single VMs or even be served from a home lab
by serving everything converged, time-to-first word can get down to 500ms
otherwise 800-1000 ms is good enough and achievable
proximity to users matters (Daily plans global regions for PipeCat cloud, currently only us-west)
conn between servers can be implemented with WebSockets

What’s next?
#

After the session I have looked at the PipeCat examples and realized it should be easy enough to run a basic voice agent with PipeCat’s SmallWebRTCTransport on a virtual server hosted in Europe and then switch the transport and deploy it to production on PipeCat Cloud.

I will probably try that to see if the latency between US based PipeCat cloud and users in Europe is low enough for a good user experience.

An Overview of the Voice AI Landscape (Session Notes)

Thu, 08 May 2025 00:00:00 +0000

I’m so happy to be part of the Voice Agents Course by Kwindla and swyx. Yesterday, Kwindla kicked it off with an overview of the voice AI landscape. The pace, insights, and questions from the audience were just great.

Here are my personal notes, probably incomplete and maybe not always correct. For a more authoritative overview of the Voice AI landscape, check out their free online book Voice AI & Voice Agents - An Illustrated Primer.

Voice AI has highly valuable use cases with actual real business value.
Benefits:
- Today: lower cost.
- Soon: Better. (peak load response, better answers than most humans can give)
RAG is still important
Some challenges:
- latency
  - measure regularly end-to-end from/to clients,
  - record conversation with mic,
  - visually look at gaps in waveforms,
  - test calls from different regions and cell phone providers
  - aim for 800ms, tough but possible to hit with hosted inference,
  - very optimized/limited deployments can hit 500ms with quality compromises
  - 1000ms is not uncommon (still makes users happy)
- turn detection
  - from VAD to semantic
  - OpenAI shipped a good text mode TDM
  - Gemini flash in audio is ok, but needs to run as parallel flow (“greedily”)
  - LifeKit vs. PipeCat is text vs. audio / end of speech, no audio cues vs. with audio cues
  - both a hard ML challenge and important for experience
- interruption handling
- context management
- function calling, tool use
- sripting, instruction following
standard architecture
- today: 3 models (STT - LLM - STT), easier to achieve robust results with LLMs in text mode
- future probably converged
- 3 models, because
  - LLMs’ text mode is their mode
  - we ride on the edge what the best models can do
  - today, we need to use their best mode
- other use cases, like language learning, can better leverage the benefits of speech-to-speech (1 model)
models used (speed and quality, time-to-first token/byte more important than tokens/s)
- STT:
  - Deepgram
  - Whisper (optimized for streaming, original not built for streaming), down at 400
  - Gladia (for non-english languages)
- LLM:
  - GPT-4o
    - still the workhorse, still more than 4.1
    - big model changes take work and good evals (nobody has good ones),
    - models usually not optimized for voice AI,
    - not yet better results
    - 4o-mini was cheaper but slower and worse for tool/function
    - note from a fellow student: 4.1-mini might be interesting
  - Gemini 2.0 Flash
    - very good, fast, cost efficient
    - best audio model for voice AI, today (gemini in audio input mode for voice-to-voice)
    - multi-lingual input is ok, output not so much (use English)
  - mid-sized open models get better, esp. fine-tuned Llama4 (upcoming OpenPipe finetuning session)
  - additional notes about LLM use in voice AI:
    - reliable function calling/tool use is mostly a question of how to prompt 4o against Gemini
    - llama can get there
    - evals are important as ever
- TTS:
  - PlayAI, Grok
  - OpenAI, Google
  - Cartesia
  - Rime
  - Elevenlabs
- network transport:
  - why network? We don’t get capable enough models on mobile or laptop (yet),
  - hybrid architectures might be relevant, already
  - telephone is a great transport for voice AI, too
    - PSTN is with a phone number (eg. from Twilio)
    - SIP is for interconnectivity with digital telephony infra
  - WebSockets are not good for real time client-to-server audio (and video), but ok for server-to-server audio
  - WebRTC is best, complex, but PipeCat supports it ootb, local for testing, in the cloud offering for production,
- Voice AI building blocks in 2025
  - evals - huge topic
  - hosting and scaling - very different (if you love k8s, your topic)
  - workflow/multi-agent/state machines
  - “perfect” speech
    - LLMs in text mode have passed the turing test
    - not quite reached the point for real-time audio recognition and generation
    - eg. accurately recording email, postal addess
  - human-like turn detection
    - very important for qualitative experience
    - fun and hard ML problem
- what’s next?
  - speech to speech models
  - realtime video
  - programming with voice
  - voice as universal user experience
  - LLM as a judge
orchestration and flows questions
- voice in text out can be done with PipeCat aggregator (no great example, yet)
- accents and voice models: Gladia input, PlayAI output
- tip for Gemini: if you want accents, specify your country.
- how to solve cold-start?
  - Daily solved it for us
  - own infra: Combine optimized startup and warm capacity
- faster responses:
  - greedily inference/speculative before turn-detection
  - helpful but more costly
- multi-speaker:
  - hard, no great solution, so far.
  - training data doesn’t map well to multi-person/agent conversation
  - challenge: know when models should or shouldn’t respond (emit a no-response-token)
- observability tooling (multiple vendors, eg. Coval, will be in Discord and hold sessions)

Even after the first session, I can already say: If you are interested in Voice AI: Take the course!.

The Fast Solopreneur

Thu, 08 May 2025 00:00:00 +0000

In The Batch 300, Andrew Ng shared some insights about the importance of speed for startups and how to move fast as a startup. I think they apply well to ideas for a fast solopreneur:

Focus on one idea,
Code prototypes intuitively,
Be creative about getting user feedback quickly (it doesn’t have to scale),
Pivot quickly but stay within your domain,
Use a KISS stack you know, and
Learn AI technology well.

Here is a summary of Andrew’s original perspective:

Focus on a concrete idea (don’t get distracted)
Switch quickly to another hypothesis when data shows that the original hypothesis is flawed
Trust a domain expert’s gut instinct
Build and test prototypes quickly with AI-assisted coding
Be fast at getting user feedback (this becomes the bottleneck and the competitive advantage)
Know the technology well (a deep understanding of what AI is good at and what it isn’t saves time and avoids dead ends)

Installing Docker on Raspberry PI

Mon, 05 May 2025 00:00:00 +0000

How to Install Docker and Docker Compose on Raspberry Pi OS (rootful and rootless).

A note on the hidden complexities of WebSockets

Sat, 25 Jan 2025 00:00:00 +0000

AI Apps are often expected to be realtime. On the web, realtime communication can be implemented with WebSockets. I’ve started with WebSockets to create chatbots and other live-updated interfaces, but then switched to SSE and now mostly follow these rules of thumbs:

Use WebSockets for server-to-server communication.
Use SSE for server-to-client communication.
If you still want or need to use WebSockets for server-to-client communication, add a fallback to SSE.
If you need realtime voice/video server-to-client communication, use WebRTC.

If you still want or need to build a websocket service, please read Atul Jalan’s blog post about The Hidden Complexity of Scaling WebSockets. It’s a quick read and captures important lessons to keep in mind when working with WebSockets.

All of his lessons are important, even when working not at scale. Here is a quick summary:

Downtimeless deployments are much more involved than in HTTP services.
Establish a good message schema, eg. 2 byte prefixes and single character field delimiters.
Use heartbeats to detect dead connections - both ways.
Have an http fallback as WebSockets are often blocked. Usually: Server sent events (SSE) for server to client communication and simple requests for client to server.
More, like standard tooling (rate limiting, validation, error handling), no caching at the edge, per-message authentication.

Standard framworks like Django, FastHTML, and Quart (an async Flask clone) have good basic support for WebSockets, but don’t really help dealing with their hidden complexities. I hope, frameworks will level up a bit, as this is mostly “undifferentiated heavy lifting”.

RTX 5090 for Local AI

Tue, 26 Nov 2024 00:00:00 +0000

A look at the NVIDIA RTX 5090 specs for local LLM inference.

ChatGPT can use information from internal sites

Tue, 05 Nov 2024 00:00:00 +0000

The ChatGPT search feature can also search and use information from intranet websites.

AI as Repository of Human Knowledge

Wed, 16 Oct 2024 00:00:00 +0000

An interesting statement by Yann LeCun.

Deploy a Static Site on Dokku

Thu, 03 Oct 2024 00:00:00 +0000

This article describes how to deploy a static site on Dokku, including activating Let’s encrypt signed certificates for the domain.

Configure DNS
#

A @ 7200 $DOKKU_SERVER_IPv4
A * 7200 $DOKKU_SERVER_IPv4
CNAME www 7200 $APP_DOMAIN
AAAA @ 7200 $DOKKU_SERVER_IPv6
AAAA * 7200 $DOKKU_SERVER_IPv6

Create a git repository
#

Create a folder and turn it into a git repository:

mkdir static-dokku
cd static-dokku
git init .

Add an index.html file with the desired content:

echo "hi" > index.html

Create an empty .static file. This tells Dokku that the repository contains a static site. Dokku then uses the Nginx builder to serve it.

touch .static

Commit the changes to git:

git add .
git commit -m "initial commit"

Create an app on the Dokku host and git push to deploy the app
#

Use these commands to create, configure, and deploy the site:

export DOKKU_HOST=<your dokku server>
export APP_DOMAIN=<your app domain>
export APP_NAME=<your app name>
ssh dokku@$DOKKU_HOST apps:create $APP_NAME
git remote add dokku dokku@$DOKKU_HOST:$APP_NAME
git push dokku main
ssh dokku@$DOKKU_HOST domains:add $APP_NAME $APP_DOMAIN
ssh dokku@$DOKKU_HOST letsencrypt:enable $APP_NAME

That’s it. The site is now live at your app domain.

Deploy updates
#

Updates can be deployed by checking them into git and running this command:

git push dokku main

Serving from sub directory
#

Serving files from a subdirectory (eg. public) is possible by setting the NGINX_ROOT environment variable of the app on the Dokku host:

ssh dokku@$DOKKU_HOST config:set $APP_NAME NGINX_ROOT=/app/www/public

Building the site during deployment
#

If you want to build the site during deployment, you can use multiple buildpacks, for example by adding a .buildpacks file to the root directory of the repository.

This way, you can use the first buildpack for building and the second fro serving.

Another, even more flexible option is to use Dockerfile deployments with multi-stage builds.

Dokku SSH Alias

Thu, 03 Oct 2024 00:00:00 +0000

A quick tip to make working a bit easier.

Launch VS Code from the Command Line

Sat, 28 Sep 2024 00:00:00 +0000

A quick tip to make your life a bit easier with VSCode and Python.

Deploying a SaaS Pegasus Based Django App on Dokku

Sun, 22 Sep 2024 00:00:00 +0000

In this post I’ll share the steps I did to deploy a SaaS Pegasus bootstrapped Django app (Scriv from the SaaS Pegasus marketplace to be precise) with Celery, Redis and Postgres on a Dokku host. There were some hickups along the way, but I believe, when you go step-by-step, it is quite straightforward.

The Furo Sphinx Theme Looks Good with Small Documentations

Sat, 10 Aug 2024 00:00:00 +0000

A quick note about the Furo theme for the Sphinx documentation system.

A Script to Export Models from Ollama

Tue, 28 May 2024 00:00:00 +0000

A workaround for transferring models to air-gapped Ollama instances.

Ollama on Windows

Sun, 18 Feb 2024 00:00:00 +0000

A tutorial and video about installing and using Ollama and OpenWebUI on Windows.

Summarizing Large Texts with LLMs and a Tree of Summaries

Sun, 04 Feb 2024 00:00:00 +0000

How to summarize texts with LLMs that are too large for their context window.

Hands-on ChatGPT in Excel (Book)

Mon, 18 Dec 2023 00:00:00 +0000

Example Excel workbooks for a book about using OpenAI models in Excel Formulas.

ChatGPT Image Inputs Use Cases

Wed, 25 Oct 2023 00:00:00 +0000

Practical use cases of ChatGPT’s / GPT-4’s image input feature.

Prompting Small LLMs.

Tue, 10 Oct 2023 00:00:00 +0000

What are small LLMs good at and how to get good results from small LLMs.

Fetching Data from REST APIs with Python and httpx

Sun, 15 Jan 2023 00:00:00 +0000

How to install and use httpx.

What is new in Excel in Q1 2022

Sun, 24 Apr 2022 00:00:00 +0000

Microsoft keeps adding new exciting features to Excel all the time. Here is an overview of new features added in the first quarter of 2022.

For me, TEXTSPLIT is the best new Excel feature of this quarter. With it you can split text that is eg. formatted as CSV into multiple rows and columns. This is extremely helpful if you want to use ChatGPT in Excel.

I base this on the Excel Blog. Please drop me a mail if I missed something. As Microsoft rolls out new features gradually, not everything might be available for you, right now.

3 New Excel Functions for Text Extraction
#

Three new functions make it easier to extract text from a cell:

TEXTBEFORE: Returns text that’s before delimiting characters
TEXTAFTER: Returns text that’s after delimiting characters
TEXTSPLIT: Splits text into rows or columns using delimiters

11 New Excel Functions for Dynamic Array Manipulation
#

Dynamic arrays have been the break trough of array formulas in Excel. Eleven new functions make working with dynamic arrays even more fun:

Functions to combine arrays:

VSTACK - Stacks arrays vertically
HSTACK - Stacks arrays horizontally

Functions to shape arrays:

WRAPROWS - Wraps a row array into a 2D array
WRAPCOLS - Wraps a column array into a 2D array
TOROW - Returns an array as one row
TOCOL - Returns an array as one column

Functions to resize arrays:

TAKE - Returns rows or columns from an array start to end
DROP - Drops rows or columns from an array start to end
CHOOSEROWS - Returns the specified rows from an array
CHOOSECOLS - Returns the specified columns from an array
EXPAND - Expands an array to the specified dimensions

Improvements in Excel for the Web
#

Conditional Formating Pane - Microsoft enhanced the user experience for creating and managing conditional formatting rules in Excel for the Web. Now you can use a pane to inspect and manage the conditional formatting rules.
Function Library - Microsoft added the function library to the formula menu. Now it is more consistent with the desktop experience.
New filter menu - The filter menu has been enhanced to make filtering data more convenient.
Insert slicer - Now you can insert slicers for PivotTables and PivotCharts in Excel for the Web, too.
Insert online pictures - With this feature, you can add images from Microsoft Stock images and from Bing search results to your worksheets.

Improved Name Manager in Excel for Mac
#

The Name Manager in Excel for Mac was not on par with Excel for Windows. Not anymore. With this update you get all the capabilities you might have been missing.

LAMBDA and LAMBDA helper functions are Generally Available for Production
#

In February 2022, Microsoft announced general availability for LAMBDA and the LAMBDA helper functions. With LAMBDA you can turn Excel formulas into custom functions. This revolutionizes how you can build formulas in Excel. Along with the GA Microsoft added an Avanced Formula Environment. This gives you a pane to edit more complicated Excel formulas in a proper editor. In addition, you can now use function tool tips and auto-completion with self-defined functions. In practice, your LAMBDA functions now look and feel like an native Excel function. Microsoft also increased the recursion limit by 16x.

Charting on the Web Feature Updates
#

Charting is a powerful tool in Excel to explore, analyze and present data. Charting in Excel for the Web was more like a viewer with only few editing capabilities. This has now changed. Microsoft has added a Chart Format Pane and On-Chart Interactions to Excel for the Web. You an also now add and remove chart elements and create charts from non-adjacent cell ranges.

AutoComplete for Dropdown Lists in Excel for Windows
#

AutoComplete for dropdown lists matches the string you type in a dropdown list cell with words from items in the dropdown list. It then shows only the matching items. As you type more characters, the list gets smaller and smaller. It matches words anywhere in the list item string.

This enhancement greatly speeds up data entry in fields with large lists. It also enables new use cases for this kind of data entry.

At time of writing (April 2022) it is only available in the Beta Channel. As an upcoming enhancement before releasing for production, Microsoft plans to add excluding duplicates from the drop-down.

Excel 4.0 (XLM) Macros are now Restricted by Default
#

You can now enable or disable Excel 4.0 (XLM) macros in the Excel Trust Center for enhanced security.

Posts on Mitja Martini

Senior means simpler

Export ChatGPT Conversations as Markdown

Hardware for local coding models is still affordable. For how long?

MCP and A2A Attack Vectors for AI Agents

Try vibe coding (again)

The Van Halen Test can check if an agent knows it's context

From 8 Lines with Dokku to 200 with Kubernetes – Why I'm Still Switching

What Takes Time in Vibe Coding

MCP in ChatGPT Developer Mode Beta

Activating developer mode #

Creating an MCP connection #

Using an MCP connection #

Current Limitations #

Trying to make sense of it #

Claude Code in Devcontainers

Tipps for Migrating to Hugo

Evals for Voice Agents (Session Notes)

Coding Agents as Slot Machines

Pipecat Cloud Latency for EU Users

K/V Cache Quantization in Ollama

Activating K/V cache quantization in Ollama #

Benefits and Drawbacks #

Example numbers #

Use cases #

More Info #