The DockerHub images link back to my GitHub, and I’m using the auto-build feature so that when I update the Dockerfile and scripts to build the software, it gets rebuilt over at DockerHub automatically.

Here’s an example of one of the builders for W1HKJ’s FlDigi software suite. Simply run the container volume mounting in some working directory to /opt/source in the container, and the build script finds the latest source, downloads, untars, and builds it, spitting out the results in your working directory.

jacob@jacob-aspire:/tmp/fldigi$ docker run --rm -v $PWD:/opt/source jacobcalvert/fldigi-build:latest 
Building fldigi-4.1.18
--2021-05-26 12:39:39--  http://www.w1hkj.com/files//fldigi/fldigi-4.1.18.tar.gz
Resolving www.w1hkj.com (www.w1hkj.com)... 143.95.246.118
Connecting to www.w1hkj.com (www.w1hkj.com)|143.95.246.118|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4847091 (4.6M) [application/x-gzip]
Saving to: 'fldigi-4.1.18.tar.gz'

     0K .......... .......... .......... .......... ..........  1%  736K 6s
    50K .......... .......... .......... .......... ..........  2% 1.53M 5s
   100K .......... .......... .......... .......... ..........  3% 2.52M 4s
   150K .......... .......... .......... .......... ..........  4% 27.7M 3s
   200K .......... .......... .......... .......... ..........  5% 3.33M 2s
   250K .......... .......... .......... .......... ..........  6% 2.67M 2s
   300K .......... .......... .......... .......... ..........  7% 28.3M 2s
   350K .......... .......... .......... .......... ..........  8% 8.94M 2s
   400K .......... .......... .......... .......... ..........  9% 9.29M 2s
   450K .......... .......... .......... .......... .......... 10% 7.04M 1s
   500K .......... .......... .......... .......... .......... 11% 4.42M 1s
   550K .......... .......... .......... .......... .......... 12% 8.31M 1s

  <<<<<<<<<<<<<<<<<<<<<<<< snipped for brevity >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

make[1]: Nothing to be done for 'all-am'.
make[1]: Leaving directory '/opt/source/flrig-1.3.54'
Done!!
jacob@jacob-aspire:/tmp/fldigi$ ll
total 11224
drwxrwxr-x  5 jacob jacob    4096 May 26 13:05 ./
drwxrwxrwt 18 root  root     4096 May 26 13:05 ../
drwxr-xr-x  9 jacob jacob    4096 May 26 13:05 fldigi-4.1.18/
-rw-r--r--  1 root  root  4847091 Jan 29 06:50 fldigi-4.1.18.tar.gz
-rw-r--r--  1 root  root  4847091 Jan 29 06:50 fldigi-4.1.18.tar.gz.1
drwxrwxr-x  7 jacob jacob    4096 May 26 12:45 flmsg-4.0.17/
-rw-r--r--  1 root  root   876560 Sep  8  2020 flmsg-4.0.17.tar.gz
drwxr-xr-x  7 jacob jacob    4096 May 26 12:46 flrig-1.3.54/
-rw-r--r--  1 root  root   891644 Feb  3 08:35 flrig-1.3.54.tar.gz

The post Docker Builders for Easy Updates appeared first on Jacob N Calvert.

The External Connector

The mini DIN connector on the back has a fairly standard pinout as shown below. Note it says 4800bps max for the DATA IN. I want to find out why!

In the service manual, I start by locating pin 1 on the DIN connector.

Internal Circuitry

Next I trace this to a pair of 2-1 muxes.

These muxes are operated together (that is, their control signal is the same signal). From my investigation, this is the mux pair that is activated when selecting the TNC operation for the radio. The right-most mux, IC114, selects between MIC IN audio from the mic port, through a series of filters, or the TNC port we’re interested in. The left-most mux, IC112, selects which TX generator chip we’re going to be using. The path I’ve highlighted in red is the TNC audio input path I’m trying to trace.

Transmission Modulation

Next the signal travels to the MICIN pin on a chip with designator IC109, and part number BK4811B.

This is where the magic of modulation happens, and where I ultimately found the reason for only supporting up to 4800bps. Scouring the web for a datasheet for this chip, I found a PDF I could download. Checking the relevant sections of the chip’s documentation, I found the culprit, screenshot below:

The audio path into the chip is shown in the blocks at the bottom. The bullet points 6 & 7 tell the details of why 9600bps isn’t usable. The low-pass and high-pass blocks cannot be bypassed on this chip, and the maximum achievable width is ~2.7k, which is not enough for 9600bps. You can turn off the compander and pre-emphasis, but there’s no way to expand the LPF->HPF network bandwidhth. To run 1200-4800bps you really only need ~1kHz, but 9600bps needs to have much more.

Wrapping Up

Now that I am satisfied that the radio can *not* in fact TX 9600bps (but can receive it, perhaps another analysis in another post to come), I will get on with getting DireWolf setup for APRS, and FlDigi for other fun digital modes.

The post Why the Alinco DR-735T can’t transmit at 9600bps appeared first on Jacob N Calvert.

I have a dual-band handheld radio for the 2m and 70cm bands. The so-called rubber-duck antenna that comes with it performs ok but it isn’t ideal. I could get into my local repeaters which are about 10mi away with enough power to break the squelch, but my audio was weak and quality was poor. Naturally, I decided I should put up an antenna!

To Build or to Buy?

You can buy antennas on the web or at local shops like my local (and fantastic) GigaParts, however in true geeky-nerd style, I have to DIY an antenna to feel I truly understand what is going on.

Self-Education

I’ll be the first to admit, prior to starting this DIY antenna journey, I didn’t have a great understanding of how antenna theory related to an actual antenna. I understood the general 1/2 wave, 1/4 wave relation to the designed frequency of an antenna, but until digging in, I didn’t get the math behind it. I found this website to be invaluable for a plain-language (plain to someone with a bit of an engineering tilt) introduction to the core concepts and how to apply them.

What kind of antenna should I build?

This was the next question I needed to answer. My antenna only had a few real requirements, namely:

1. Support the 2m band
2. Support the 70cm band
3. Be outdoor-mountable
4. Be home-manufacturable

This list ruled out a standard di-pole, because although I have learned you can “load” a dipole to make it multi-band, I didn’t feel I had the mounting capability at present for that. I also ruled out a ladder-line j-pole, because I couldn’t find ladder-line anywhere near me. It used to be more common I suppose when OTA TV used it for their antennas, but no more. I ruled out a Yagi also, based on the complicated layout, and I didn’t necessarily want a directional antenna. After scouring the web, I settled on a standard J-pole.

J-Pole: what is it and how does it work?

If you navigate to the Wikipedia entry for J-pole antenna , you will notice there are many “form-factors,” but they all follow the similar J shape, hence the name. But looking at the typical design, I couldn’t understand how the feed point (which separated only by a short distance) doesn’t just ruin the antenna performance. Take a look at the image below, credit ZyMOS.

I found the most useful explanation in a YouTube video (an aside: you can anything on YT these days) which I will link.

How it works, condensed version

A little prerequisite knowledge will help the understanding of how this antenna works.

1. The impedance in the middle of a 1/2 wave dipole is very low
2. The impedance at the ends of a 1/2 wave dipole is very high
3. The impedance somewhere between “very low” and “very high” is our magical 50Ω
4. We can end-feed a 1/2 wave antenna, but the impedance is very high (as seen in item #2)

Basically, the bottom U part of the j-pole is where the feed-point is adjusted to 50Ω (or thereabouts), and at the ends the impedance is very high, which is perfect to end-feed our 1/2 wave antenna (the long part of the J). As a result of this configuration, it really doesn’t matter which “leg” of the antenna is tied to shield vs. conductor for the feed-point.

But this is still for a single band, right?

Yes, but conveniently, the “long” side of the j-pole for 70cm band is pretty darn close to the “short” side of the j-pole for the 2m band, and since it doesn’t matter which element is our “radiating” element, we can essentially reuse one of our elements. See the rough diagram below for an example of this.

Let’s Build It!

So I had armed myself with the knowledge (at least the theory) and now I just needed to create a plan. Like any good engineer, I looked around the web for any prior work in this area (no reason to reinvent the wheel!) and much to my delight, someone had posted a PDF of an easy build process for such an antenna. Following this guide, I was able to construct my first dual-band j-pole.

Tuning the antenna

This step is very important, and can be made easier by the use of SWR meters or in my case a Vector Network Analyzer. I purchased the NanoVNA (wonderful tool, works like a charm) and was able to adjust the elements of my new antenna to achieve a great SWR and feed-point impedance in the 2m band, and pretty good SWR and impedance in the 70cm band. See captures below.

2m Tuning Results

70cm Tuning Result

Some notes on the 70cm performance

You’ll notice the oscillatory nature of the SWR and impedance in the 70cm band. I do not completely understand this, and may return to tweak the antenna a little more. I get decent performance in several “buckets” of the 70cm band, so it’s workable, but not ideal. I have a couple of theories as to why this is, but I don’t know for sure. If any readers have some comments on this, I’d love to hear it, please comment or send me some email!

Element Diameter

The elements in the 70cm legs of the antenna are 3/8″ rod as opposed to 3/4″ EMT conduit. I suspect that the deep V around 441MHz is the center frequency for these stubs, and the antenna just has a narrow bandwidth because the elements are small. I will likely try to build another one with all 3/4″ elements to see if the antenna is improved.

Element Smoothness

I’m not sure that this has an impact or not, but the 3/8″ rod is threaded at 24-TPI and the 3/4″ elements are smooth EMT conduit. I wonder if this interplays with the skin effect?

The End Result

I mounted this antenna up on the roof, and boy-oh-boy does it work great! I am able to be heard at least 17mi away by an APRS digipeater, which was not possible with the rubber duck antenna. I can now full-quiet the local repeaters and my voice is loud and clear. Also, mounting the antenna outside has greatly reduce computer and monitor based interference, which was a pleasant surprise. All said, I’d say this was a great learning experience and fun project to boot!

Thanks for reading!

The post J-Pole Antennas for Ham Radio appeared first on Jacob N Calvert.

To be honest, I love self-hosting tools. I think it has to do with my love of learning, but I digress. I know there are gobs of note-taking/sharing/synchronizing apps out there, many of them free, but they all lacked features, and I wanted to self-host.

What I Was Looking For

I wanted a note-taking app, with the following capabilities:

• Self-hosted
• Synchronization across devices
• E2EE (end-to-end encryption)
• Rich media note-taking ability
  • Markdown style note-taking
  • Image support
  • Code highlighting
  • Embeddable documents
  • Hyperlinks
  • Etc.
• Web-based interface
  • Desktop and Mobile capable

What I Found

After searching the interwebs for a bit I ran across Trilium and Joplin.

Admittedly, I tried Trilium first, since it boasted a Desktop-and-mobile web platform which got rid of my need for synchronization. But after a few days, I decided to move on. Trilium is an electron-based application, and it just doesn’t work as well in the browser as it does in the app. I decided I’d give Joplin a try (and I’m certainly glad I did!).

Joplin for the Win!

Joplin is pretty much everything I’ve needed for this endeavor. It even comes packed with some extra cool features like the text-based Mermaid diagramming capability. It lacks a web app, but it does have iOS, Android, and Desktop apps, which work really well. Another feature I love is nested notebooks. This allows me to keep a running daily note, in a hierarchically way. See the image below.

Another great feature is the simplicity of sync and encryption. I’ve got a WebDAV server setup which runs over HTTPS and using E2EE, everything stays nice and secure.

The post Take notes using Joplin appeared first on Jacob N Calvert.

The Problem

The standard installation procedure for Windows via USB for a good while has been to dd the installer ISO image onto a USB disk and load it up on your computer. This changed a little when UEFI and SecureBoot became the norm for consumer equipment. Now, it’s pretty common to have a EFI boot partition FAT32 formatted with the EFI/BOOT/BOOTX64.EFI and all associated installers in the same partition. A typical machine will boot up, read the EFI boot file and go on with it’s business. Here’s where the problem begins.

FAT32 and the install.wim

In recent versions of Windows 10, there’s a file named install.wim. This file contains the bulk of the ~5.4GB installation image, and is more than 4GB alone. Herein lies the problem! FAT32 formatted partitions can’t index a single file larger than 4GB (32 bits worth of bytes ya know?). This presents a large problem! How am I to install this OS when I can’t load the files onto a partition to load the installer?

OK, so now what? Doesn’t UEFI require a FAT32 formatted partition?

Well, as it turns out, UEFI doesn’t actually care what format the EFI files are in, so long as the firmware know how to read it. So I tried to pivot to another common format: NTFS.

Booting UEFI via an NTFS partition?

Indeed! But this didn’t prove so fruitful at first. You see, ASUSTek only saw fit to include a FAT32 driver for their EFI loader, so the NTFS drive with the Windows 10 installation files on it just was not recognized by the firmware.

Stuck!

At this point, I was frustrated and was asking all sorts of questions like:

• How does ASUS load their image? Are they mass cloning drives and not actually installing? (Probably)
• Does this mean there’s no hope for a clean install of Windows 10 for dear wife? (Seems so)
• Why would they only include a FAT32 driver? (Who knows!?)

After a long while of Google-Fu (ok, actually DuckDuckGo), I stumbled upon this wonderful saint of an engineer’s GitHub repo.

Solution: UEFI-NTFS

This repo contains the answer to my prayers. You can read all the details at the repo, but here’s the solution in a nutshell.

Create a drive with two partitions, one partition FAT32 formatted and loaded with the UEFI-NTFS EFI goodies from the linked repo.

The other partition is NTFS formatted, and contains the desired over-4GB files we want to use.

The machine firmware will ignore the NTFS partition on this drive and launch the EFI booter on the FAT32 partition. This EFI booter will in turn load an NTFS driver, hunt down the NTFS partition adjacent, and execute the /EFI/BOOT/BOOTX64.EFI from there, chainloading the Windows 10 installation.

It worked like a charm! Within minutes the Windows 10 installer was running, and I was a much happier camper, and dear wife has a much faster OS to use due to this new SSD.

The post Who knew installing Windows could be so hard? appeared first on Jacob N Calvert.

I’ll give an example of what I mean by “internalize and restate it”.

To understand orbital motion (things like planets, moons, satellites), the mathematics to get a precise understanding of the shape and speed of an object in motion (Kepler’s First and Second Laws) can be fairly intimidating. For me, the breakthrough in highschool Physics class was comparing the speed and motion of an orbiting body to paddle ball toy. If you sling the ball part of a paddle ball toy while fixing the paddle part flat on a table, the elastic in the string will cause the ball to move slower the further away from the paddle it gets, and as it returns the elastic causes it to move faster.

Now this example doesn’t compare apples to apples because the elastic force in my example and gravitational forces in the physical world mathematically do not work the same way, but it’s an easy way to visualize the concept. PS: for a neat visualization with computer graphics, see here.

Back to the calculus now: I have found a book from the early 1910s called Calculus Made Easy, by Silvanus Thompson. Thompson’s approach to demystifying the concepts in basic calculus are the most straightforward I’ve ever read. He uses simple, relatable concepts to help the reader internalize and restate the problems in a domain that most have a keen understanding of, and as a result, the concepts become much less scary and much more useful! This book is freely available online as a PDF from Project Gutenbergm and a dedicated website. Both linked below for convenience.

Whether you’re learning calculus for the first time, need a basic refresher, or just want to reinforce your understanding of the concepts, I highly recommend giving it a look.

Project Gutenberg Link

Website Link

The post Calculus Made Simple appeared first on Jacob N Calvert.

This hotly debated topic has been around for decades, and it is just as alive today as it was 28 years ago. The truth is, there are fundamental differences in the theory which drives the design of a monolithic kernel versus a microkernel. In this post, I will extrapolate from my knowledge of various kernel designs to explore what these two primary types are, what their features, benefits, drawbacks and implications may be. I’ll also briefly explore the extension of the theories that drive these designs into “fringe” types of kernels (such as a nanokernel).

Key Concepts

Before hopping into the discussion about the differences between these two kernel archetypes, I need to lay out some key concepts which will help bound the discussion.

Userspace vs. Kernelspace

Broadly, userspace is considered to be “outside” of the kernel. Kernelspace is “inside.” The way this is typically implemented is through the use of multiple address spaces, aided by the hardware (MMU, MPU). This usually results in all userspace applications having believing they have the whole address space to themselves, while in reality the MMU is remapping some Virtual Addresses (VAs) to the real Physical Addresses (PAs). Actions in userspace cannot (well, SHOULD not) perturb the operation of the core kernel. Contrast this to kernelspace, wherein any part of the kernel is available to poke and tinker with from code running in this space.

This separation of spaces may also use other hardware features (Rings in x86-land, Exception Levels in ARM) to further represent the distinction and privilege of code running in each space.

Examples of userspace are your typical Linux application.

Examples of kernelspace are a Linux .ko kernel module.

Kernel-User Boundary

This boundary is the line which separates the userspace and kernelspace. Crossing this boundary involves a change in privilege level, and is often accomplished by invoking a system call. Why and when we cross this boundary is a key point of interest in the discussion of kernel design since it is usually an expensive operation.

Essential Operating System Functions

For the purpose of this discussion, I consider the following to be essential to any OS for it to be of any use.

• Memory Management
  • Addresses needs of the kernel to map/unmap/remap Virtual Memories to Physical ones
• Task/Process Management
  • Addresses needs of the kernel to handle multi-tasking
• Inter-process Communication (IPC)
  • Addresses needs of kernel and applications to communicate via a common interface
• Device Drivers
  • Addresses needs of applications to communicate with the outside world
• Filesystems
  • Addresses needs of application and kernel to store/retrieve data via a common interface

Monolithic Kernels

Monolithic kernels and the essential properties which make it monolithic can be succinctly summed up in the following phrase:

Everything except the application exists in kernelspace.

In a monolithic kernel, all the essential components, and many other accessory components, live in kernelspace.

The design choice to bring all these functions and services into the kernelspace has several benefits, drawbacks, and implications.

Benefits

Since all the code that directly interacts with devices live in the same address space, moving data around is an inexpensive operation. Much of what happens in a system falls into this category, so you end up with a relatively speedy and snappy kernel implementation. For example, if two network devices are being bridged to transmit data from one network to another, and the network stacks and device drivers live in kernelspace, this operation can occur without any context change, saving time and increasing throughput.

Another benefit is that the userspace applications have a rich set of services which has a consistent interface available (usually the system call interface). This allows rich applications to be developed using a standard scheme.

Drawbacks

The same feature which is a benefit to monolithic kernels enabling speedy execution of kernelspace work, also is a drawback in some scenarios. If anything in kernelspace fails, it can potentially impact all of kernelspace. This is often addressed by having hardware enforced protections of kernelspace tasks as well as userspace tasks, which in many instances, ameliorates this issue.

Another drawback is that monolithic kernels must be updated atomically. They are typically tightly coupled and updates can be difficult to introduce.

Implications

If speed is a primary concern, monolithic kernels are good choices.

Microkernels

Microkernels are the minimalist brother to monolithic kernels. The essence of the microkernel can be summed up as:

Only the bare minimum to operate the kernel lives in kernelspace.

Nothing besides the minimal set of operations to consider the kernel operational resides in kernelspace in a microkernel. Many resources say this set is:

• Memory Management
• Process Management
• IPC

I like to add one more to the list because I feel it is a critical resource in a real system: Interrupt Management. Either way, it is easily seen that the microkernel contains fewer components running in kernelspace, hence the “micro” name.

The services that were once part of the kernelspace in the monolithic design are now run as “servers” in the userspace. These servers interact with the applications and the other servers via IPC, which is facilitated by the kernel. This of course has benefits, drawbacks and implications as well.

Benefits

One obvious benefit over the monolithic kernel is that the servers can be taken online/offline independently of the kernel and a failure in one does not induce a system-wide failure necessarily.

Drawbacks

Similarly to the monolithic kernel, the primary benefit to a microkernel also has a drawback associated. Because each server runs in userspace and relies on IPC as a primary mechanism of communications, constant context switching into and out of kernel mode means microkernels are typically slower than their monolithic counterparts.

Another drawback is that not all expected services are going to be available on a microkernel since they are effectively decoupled from kernel execution.

Implications

If service independence is most important, a microkernel is a good choice.

Key Takeaways

There are some aspects of the two primary kernel types I didn’t dig into, and I want to address them specifically below.

Security

There is a strong case for both monolithic and microkernels in the security perspective. In this realm, the truth is that both designs have strengths and weaknesses, and the kernel’s design is only one aspect of the security posture of the system.

Safety

Like security, safety has a place in both micro and monolithic kernels. The safety of each relies more on the implementation and usage of the particular kernel rather than its design.

Absoluteness of Design

In today’s software architectures, we regularly see hybrids of these two concepts where we can meld the benefits of each in a blended fashion. This is a bit of a hot topic because when the lines are blurred it is harder to define what falls into each category. For examples of hybridization, see this article.

Extension of the Concepts

By taking the concepts which drive the monolithic and particularly the microkernel design to extremes, we see some interesting outcomes. Take for instance the nanokernel. This is the extreme version of a microkernel, where additional services may not be available and the application must provide this all itself. This is tending towards being simply a hypervisor however, and may not be a microkernel at all. Another example is the exokernel where the hardware resources are basically un-abstracted completely, so the application must make all these decisions. Of note regarding exokernels, is that they have yet to be developed into a commercial-grade offering as of now, and are primarily a research concept.

The post Kernel Design: Microkernel vs. Monolithic appeared first on Jacob N Calvert.

Note: all the code, scripts, etc. are archived in a git repo at GitHub.

Diving In

As a refresher from the Types of Virtualization post, type-2 hypervisors are virtualization systems in which a Host OS sits between the Guest OS and the hardware. Sometimes, especially when the Guest OSes instruction set differs from the Host OSes instruction set, this type of hypervisors are called emulators. In this context, this type of hypervisor is well-suited for embedded systems development and test. For this deep dive, I will be focusing on the emulation side of type-2 hypervisors. (The other side of type-2 hypervisors are more like VirtualBox and are not particularly useful for embedded systems development).

QEMU as a Development Platform

QEMU is an open-source emulator which has broad community support and contains libraries for emulating a large number of targets of many different instruction sets and peripherals. This makes it a particularly useful tool for developing code for a target for which you don’t have real hardware.

Setting up a QEMU Target

Luckily, QEMU has prebuilt packages for most platforms you can simply install using the package manager of your choice. I’m going to be targeting the aarch64 (also referred to as arm64 in Linux land) architecture for emulating a virtual platform similar to the Raspberry Pi 3 (and newer). I have installed the package on my Linux Mint machine which provides qemu-system-aarch64. With this emulator available, I can invoke the emulator with the following command:

qemu-system-aarch64 -m virt -M 1G -kernel kernel.elf

This command will start up an ARMv8-A-based virtual machine, with 1GB of RAM, and load the code in kernel.elf.

The post Virtualization for Embedded Systems Series: Type-2 Hypervisors Deep Dive appeared first on Jacob N Calvert.

Here’s a breakdown: on the FT232R, the max speed I can set the adapter to is 3MHz. This has given me a transfer speed (loading via GDB) of around 3KB/s. Not too bad for small projects of only a few hundred KiB. But now my code size is approaching a ~5 MiB, the debug cycle was way too long… 5MiB @ 3KB/s = ~28mins to load… not good!

Still in my pursuit of a cheap JTAG debugger (I mean come on! It’s just a serial protocol!), I did buy something. The Adafruit FT232H breakout is exactly what I was looking for. At only $15 with Amazon Prime shipping, I was back to debugging at a reasonable speed in two days.

The FT232H breakout is about as barebones as it gets. The board breaks out untouched ACBUS0-ACBUS7 and ADBUS0-ADBUS7 to 0.10″ pitch headers. Since the FT232H has a single MPSSE channel, I can use this breakout to get faster JTAG speeds.

My configuration files from the previous post had to change a little to accommodate. I’ll break it down below in comments in the file because it’s a little bit cryptic.

#
# FTDI USB Hi-Speed to MPSSE Breakout from Adafruit
#
# This should work for any bare FT232H
#

# Setup driver type
adapter driver ftdi

# 30000 kHZ -> 30MHz
adapter speed 30000

# Using JTAG (also could be SWD)
transport select jtag

# Common PID for FT232H
ftdi_vid_pid 0x0403 0x6014

# Set sampling to allow higher clock speed
ftdi_tdo_sample_edge falling


# Layout
# On this breakout, the LEDs are on ACBUS8 and ACBUS9, can't assign them
# registers are <ACVALUE><ADVALUE> <ACCONFIG><ADCONFIG>
# so we set 0x0308 to mean only ACBUS nTRST and nSRST, ADBUS3 (TMS) asserted high
# and we set 0x000B to mean only AC3,AC2,AC0 outputs -> (TMS,TD0, TCK)
ftdi_layout_init 0x0308 0x000b

# Pins
# pin name      | func. |
# --------------|-------|
# ADBUS0        | TCK   |
# ADBUS1        | TDI   |
# ADBUS2        | TDO   |
# ADBUS3        | TMS   |
# ACBUS0        | nTRST |
# ACBUS1        | nSRST |
#---------------|-------|

# When data == oe -> pins are switched from output to input to give
# the tri state (L, H, Hi-Z) effect 
ftdi_layout_signal nTRST -data 0x0100 -oe 0x0100
ftdi_layout_signal nSRST -data 0x0200 -oe 0x0200

So now it is as simple as connecting the corresponding pins on this breakout to the RPi3, and running OpenOCD again.

With the increased throughput of this MPSSE-supported interface, I now get ~587KB/s. Much better! It only takes ~8s to load my image now.

The post Better JTAG on the Cheap with the FT232H appeared first on Jacob N Calvert.

What is it?

JTAG stands for the Joint Test Action Group, and the TAP or Test Access Port this group defined is one of the most (if not the most) common way to program and debug embedded devices and computers of all flavors. For the professional, JTAG devices are bountiful and usually not too much of a strain on the commercial budget. But for the hobbyist, things aren’t so peachy. A Segger J-Link EDU can be had for ~$70 USD shipped, but the full featured J-Link is still ~$400, which is more than I want to pay as a hobbyist.

What does it do?

The JTAG TAP port consists of a few standard signals which essentially give you complete control over the systems in the JTAG chain. The chain is exactly what it sounds like: multiple devices which support JTAG chaining can be chained up and accessed from a single JTAG port. Need that flash programmed? JTAG can do that. Need to debug that microcontroller? JTAG can do that too. Want to do both without switching out tools? Yep, JTAG can do it. See this helpful diagram at Wikipedia for a visual representation.

JTAG for the Hobbyist

Chances are that you’ve got an USB -> Serial cable or breakout board lying around somewhere in your hoard. Chances are also, that it’s based on the wildly popular FTDI FT232R or a similar FT232-esque chip which converts USB to RS232. If you’re lucky enough to have this essential piece of hobbyist equipment, you’ve got a USB->JTAG adapter waiting to be unlocked!

Enter OpenOCD

OpenOCD (Open On-chip Debugger) is a fantastic project which aims to create an open and extensible OCD solution for all, and lucky for us, this includes the hobbyist! The OpenOCD project defines interfaces between the common parts of the OCD process, such as the target board or device, the OCD device used, etc., and using these well-defined interfaces is able to create a modular system which can support many different targets and debuggers with only a configuration change. This is a simplification of how this works, but it is sufficient to understand what we want to do with it.

So back to the FT232R. This little chip can be re-configured to use the RS232 signals as bit-banged JTAG signals, and OpenOCD can drive it. Even better, once the OpenOCD agent is up and running, we can then use GDB to connect to and drive our debug efforts. This mode of operation is detailed over at their documentation site (hint: search for ft232r).

The OpenOCD tool can usually be installed with your package manager on Linux. I’m running Linux Mint, so I apt install’d openocd. When I ran the tool pointing to the ft232r config file, it complained that it was not a supported interface… so I guess we’ll build from source!

Building OpenOCD with the Right Interfaces

All we need to do is build OpenOCD from source with the right interfaces enabled and we can make it work. What follows next is my step-by-step on doing this. I’m doing this in /tmp just so I can recreate the steps I did earlier in my apps directory.

Grab the Source

Go get the source from the GitHub mirror here and drop it in a working directory. I would also read the README for any dependencies you didn’t have already on your machine.

jacob@jacob-aspire-mint:/tmp$ git clone https://github.com/ntfreak/openocd.git
Cloning into 'openocd'...
remote: Enumerating objects: 74, done.
remote: Counting objects: 100% (74/74), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 62845 (delta 36), reused 50 (delta 21), pack-reused 62771
Receiving objects: 100% (62845/62845), 24.17 MiB | 24.90 MiB/s, done.
Resolving deltas: 100% (51571/51571), done.
jacob@jacob-aspire-mint:/tmp$ 

Run the bootstrapper

This little piece of code essentially checks that your build environment is sane and you have all the tools needed to build OpenOCD. Output is below, you should come away with no errors.

jacob@jacob-aspire-mint:/tmp/openocd$ ./bootstrap 
+ aclocal
+ libtoolize --automake --copy
+ autoconf
+ autoheader
+ automake --gnu --add-missing --copy
configure.ac:26: installing './compile'
configure.ac:37: installing './config.guess'
configure.ac:37: installing './config.sub'
configure.ac:16: installing './install-sh'
configure.ac:16: installing './missing'
Makefile.am:46: warning: wildcard $(srcdir: non-POSIX variable name
Makefile.am:46: (probably a GNU make extension)
Makefile.am: installing './INSTALL'
Makefile.am: installing './depcomp'
Makefile.am:23: installing './mdate-sh'
Makefile.am:23: installing './texinfo.tex'
Setting up submodules
Submodule 'jimtcl' (http://repo.or.cz/r/jimtcl.git) registered for path 'jimtcl'
Submodule 'src/jtag/drivers/libjaylink' (http://repo.or.cz/r/libjaylink.git) registered for path 'src/jtag/drivers/libjaylink'
Submodule 'tools/git2cl' (http://repo.or.cz/r/git2cl.git) registered for path 'tools/git2cl'
Cloning into '/tmp/openocd/jimtcl'...
warning: redirecting to https://repo.or.cz/r/jimtcl.git/
Cloning into '/tmp/openocd/src/jtag/drivers/libjaylink'...
warning: redirecting to https://repo.or.cz/r/libjaylink.git/
Cloning into '/tmp/openocd/tools/git2cl'...
warning: redirecting to https://repo.or.cz/r/git2cl.git/
Submodule path 'jimtcl': checked out 'a9bf5975fd0f89974d689a2d9ebd0873c8d64787'
Submodule path 'src/jtag/drivers/libjaylink': checked out 'f73ad5e667ae8b26a52b847c603fdadaabf302a6'
Submodule path 'tools/git2cl': checked out '8373c9f74993e218a08819cbcdbab3f3564bbeba'
Generating build system...
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/config.guess'
libtoolize: copying file 'build-aux/config.sub'
libtoolize: copying file 'build-aux/install-sh'
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
configure.ac:42: installing 'build-aux/ar-lib'
configure.ac:37: installing 'build-aux/compile'
configure.ac:30: installing 'build-aux/missing'
Makefile.am: installing './INSTALL'
libjaylink/Makefile.am: installing 'build-aux/depcomp'
Bootstrap complete. Quick build instructions:
./configure ....
jacob@jacob-aspire-mint:/tmp/openocd$ 

Run the configure script

This is where we determine that we want the FTDI FT232R to be supported. We do this by adding a flag to the configure line. Output is cut short in the middle because it’s quite long, but the important part is at the end.

jacob@jacob-aspire-mint:/tmp/openocd$ ./configure --enable-ft232r 
checking for makeinfo... no
configure: WARNING: Info documentation will not be built.
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk

[ ... snip ... ]

libjaylink configuration summary:
 - Package version ................ 0.2.0-git-f73ad5e
 - Library version ................ 0:0:0
 - Installation prefix ............ /usr/local
 - Building on .................... x86_64-pc-linux-gnu
 - Building for ................... x86_64-pc-linux-gnu

Enabled transports:
 - USB ............................ yes
 - TCP ............................ yes



OpenOCD configuration summary
--------------------------------------------------
MPSSE mode of FTDI based devices        yes (auto)
ST-Link Programmer                      yes (auto)
TI ICDI JTAG Programmer                 yes (auto)
Keil ULINK JTAG Programmer              yes (auto)
Altera USB-Blaster II Compatible        yes (auto)
Bitbang mode of FT232R based devices    yes
Versaloon-Link JTAG Programmer          yes (auto)
TI XDS110 Debug Probe                   yes (auto)
OSBDM (JTAG only) Programmer            yes (auto)
eStick/opendous JTAG Programmer         yes (auto)
Andes JTAG Programmer                   yes (auto)
USBProg JTAG Programmer                 no
Raisonance RLink JTAG Programmer        no
Olimex ARM-JTAG-EW Programmer           no
CMSIS-DAP Compliant Debugger            no
Cypress KitProg Programmer              no
Altera USB-Blaster Compatible           no
ASIX Presto Adapter                     no
OpenJTAG Adapter                        no
SEGGER J-Link Programmer                yes (auto)

Build it

Finally, we build it. Again, I’m snipping the output down to size, but you should end up with an executable binary in src/ called openocd. This final step doesn’t take long (on my machine only about 45s).

jacob@jacob-aspire-mint:/tmp/openocd$ make
Makefile:4634: warning: overriding recipe for target 'check-recursive'
Makefile:4045: warning: ignoring old recipe for target 'check-recursive'
cat src/helper/startup.tcl src/jtag/startup.tcl src/target/startup.tcl src/server/startup.tcl src/flash/startup.tcl | ./src/helper/bin2char.sh > src/startup_tcl.inc || { rm -f src/startup_tcl.inc; false; }
cp src/jtag/drivers/minidriver_imp.h src/jtag/minidriver_imp.h
make  all-recursive
make[1]: Entering directory '/tmp/openocd'
Makefile:4634: warning: overriding recipe for target 'check-recursive'
Makefile:4045: warning: ignoring old recipe for target 'check-recursive'
Making all in jimtcl
make[2]: Entering directory '/tmp/openocd/jimtcl'


[ ... snip ... ]

libtool: link: ranlib src/.libs/libopenocd.a
libtool: link: rm -fr src/.libs/libopenocd.lax src/.libs/libopenocd.lax
libtool: link: ( cd "src/.libs" && rm -f "libopenocd.la" && ln -s "../libopenocd.la" "libopenocd.la" )
depbase=`echo src/main.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
gcc -DHAVE_CONFIG_H -I.   -I./src -I./src -I./src/helper -DPKGDATADIR=\"/usr/local/share/openocd\" -DBINDIR=\"/usr/local/bin\" -I./jimtcl -I./jimtcl  -Wall -Wstrict-prototypes -Wformat-security -Wshadow -Wextra -Wno-unused-parameter -Wbad-function-cast -Wcast-align -Wredundant-decls -Werror -g -O2 -MT src/main.o -MD -MP -MF $depbase.Tpo -c -o src/main.o src/main.c &&\
mv -f $depbase.Tpo $depbase.Po
/bin/bash ./libtool  --tag=CC   --mode=link gcc -Wall -Wstrict-prototypes -Wformat-security -Wshadow -Wextra -Wno-unused-parameter -Wbad-function-cast -Wcast-align -Wredundant-decls -Werror -g -O2   -o src/openocd src/main.o src/libopenocd.la  ./jimtcl/libjim.a  -ldl 
libtool: link: gcc -Wall -Wstrict-prototypes -Wformat-security -Wshadow -Wextra -Wno-unused-parameter -Wbad-function-cast -Wcast-align -Wredundant-decls -Werror -g -O2 -o src/openocd src/main.o  src/.libs/libopenocd.a -lusb-1.0 -lm ./jimtcl/libjim.a -ldl
make[2]: Leaving directory '/tmp/openocd'
make[1]: Leaving directory '/tmp/openocd'

Testing OpenOCD

Now that we have a binary, we need to test it out and see if it works. My target for today is a Raspberry Pi 3 B. We first need to gather some info about our target, i.e., the Raspberry Pi 3. We need to know which pins on the GPIO header correspond to which JTAG signals.

First let’s grab the BCM2835 pinout reference. (Side note: we use the BCM2835 reference because there is no BCM2837 reference, and it seems the pinouts are roughly the same.) This will tell us what each GPIO pin does. Some of the GPIOs are only designated for one singular function, whereas many other GPIOs are multi-function, and their function is software programmable. These alternate functions are designated ALT on the reference material. I used the reference over at e-Linux because I find the table easy to read. Searching the table, we find the JTAG signals we’re interested in (TRST, TCK, TMS, TDI, TDO) are assigned in the range of GPIOs 22-27 for the ALT4 function. Next we want to pull up the schematic of our Raspberry Pi 3 B, and see what each of those map to on the GPIO header. I’ve created the table below to keep things straight.

Once we’ve got these figured out we’ve got to prep the Pi for JTAG usage. This is the simplest part in the whole setup. On the Pi’s SD card, add the following line to the config.txt.

enable_jtag_gpio=1

Finally, we’re ready to hook up our JTAG debugger. Looking in the documentation for the OpenOCD FT232R configuration, we find the following table which shows us which RS232 signals correspond to which JTAG signals. We need to (powered down of course!) hook up our FT232R module/cable and Pi accordingly, and don’t forget the ground wire!

    - RXD(5) - TDI
    - TXD(1) - TCK
    - RTS(3) - TDO
    - CTS(11) - TMS
    - DTR(2) - TRST
    - DCD(10) - SRST 

Once things are hooked up, we need to setup our configuration for the OpenOCD tool. The default config from the ft232r.cfg is sufficient for the interface, but I wanted to add one item and to get us debugging we need to tell OpenOCD how to connect to the Pi and its four cores. I’ve based my rpi3b.cfg config off of a fellow GitHub contributor’s config but made some modifications.

ft232r.cfg

adapter driver ft232r
adapter speed 3000
ft232r_restore_serial 0x15

I modified the ft232r.cfg to increase the adapter speed to 3M (3000 kHz) and to restore the port config when OpenOCD is done so the adapter can be used as a serial device again.

rpi3b.cfg

transport select jtag

adapter speed 3000

reset_config trst_and_srst

jtag_ntrst_delay 500

if { [info exists CHIPNAME] } {
  set _CHIPNAME $CHIPNAME
} else {
  set _CHIPNAME rpi3
}

if { [info exists DAP_TAPID] } {
   set _DAP_TAPID $DAP_TAPID
} else {
   set _DAP_TAPID 0x4ba00477
}

jtag newtap $_CHIPNAME tap -irlen 4 -ircapture 0x1 -irmask 0xf -expected-id $_DAP_TAPID -enable
dap create $_CHIPNAME.dap -chain-position $_CHIPNAME.tap

set _TARGETNAME $_CHIPNAME.a53
set _CTINAME $_CHIPNAME.cti

set DBGBASE {0x80010000 0x80012000 0x80014000 0x80016000}
set CTIBASE {0x80018000 0x80019000 0x8001a000 0x8001b000}
set _cores 4

for { set _core 0 } { $_core < $_cores } { incr _core } {

    cti create $_CTINAME.$_core -dap $_CHIPNAME.dap -ap-num 0 \
        -ctibase [lindex $CTIBASE $_core]

    target create $_TARGETNAME.$_core aarch64 \
        -dap $_CHIPNAME.dap -coreid $_core \
        -dbgbase [lindex $DBGBASE $_core] -cti $_CTINAME.$_core

    $_TARGETNAME.$_core configure -event reset-assert-post "aarch64 dbginit"
}

Putting it All Together

Now we have all the pieces we need to debug using our JTAG adapter. Let’s begin!

jacob@jacob-aspire-mint:/opt/apps/openocd$ sudo ./src/openocd -f ./ft232r.cfg -f ./rpi3b.cfg 
Open On-Chip Debugger 0.10.0+dev-01047-g09ac9ab1 (2020-02-04-09:11)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : only one transport option; autoselect 'jtag'
FT232R restore serial: 0x0015 (enabled)

Warn : Transport "jtag" was already selected
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : clock speed 3000 kHz
Info : JTAG tap: rpi3.tap tap/device found: 0x4ba00477 (mfg: 0x23b (ARM Ltd.), part: 0xba00, ver: 0x4)
Info : rpi3.a53.0: hardware has 6 breakpoints, 4 watchpoints
Info : rpi3.a53.1: hardware has 6 breakpoints, 4 watchpoints
Info : rpi3.a53.2: hardware has 6 breakpoints, 4 watchpoints
Info : rpi3.a53.3: hardware has 6 breakpoints, 4 watchpoints
Info : Listening on port 3333 for gdb connections
Info : Listening on port 3334 for gdb connections
Info : Listening on port 3335 for gdb connections
Info : Listening on port 3336 for gdb connections

Now we’re cooking! You can connect GDB up to each core, debug that way, or use the OpenOCD interface, see below:

OpenOCD Interface

acob@jacob-aspire-mint:/tmp/openocd$ telnet localhost 4444
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
> halt
rpi3.a53.3 cluster 0 core 3 multi core
target halted in AArch64 state due to debug-request, current mode: EL3H
cpsr: 0x000003cd pc: 0x20001c
MMU: disabled, D-Cache: disabled, I-Cache: disabled
> targets
    TargetName         Type       Endian TapName            State       
--  ------------------ ---------- ------ ------------------ ------------
 0  rpi3.a53.0         aarch64    little rpi3.tap           running
 1  rpi3.a53.1         aarch64    little rpi3.tap           running
 2  rpi3.a53.2         aarch64    little rpi3.tap           running
 3* rpi3.a53.3         aarch64    little rpi3.tap           halted

> reg
===== Aarch64 registers
(0) x0 (/64): 0x0000000000000003 (dirty)
(1) x1 (/64): 0x00000000C1000000
(2) x2 (/64)
(3) x3 (/64)
(4) x4 (/64)
(5) x5 (/64)
(6) x6 (/64)
(7) x7 (/64)
(8) x8 (/64)

[ ... snip ... ]

GDB Interface

jacob@jacob-aspire-mint:/tmp/openocd$ aarch64-none-elf-gdb
GNU gdb (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 8.3.0.20190709-git
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=aarch64-none-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.linaro.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) target remote :3333
Remote debugging using :3333
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
0x0000000000200028 in ?? ()
(gdb) si
0x000000000020002c in ?? ()
(gdb) si
0x0000000000200074 in ?? ()
(gdb) si
0x0000000000200078 in ?? ()
(gdb) 

Wrapping Up

Now we have a JTAG solution that can debug many, many flavors of processors and microcontrollers given the right configuration. It becomes even more useful if your FT232R module supports selecting the logic level voltage like mine does. See below for some pros and cons of this approach.

Pros

This is a quick and dirty way to get a JTAG adapter at no cost if you’ve already got these FTDI chips laying around. It’s quick and fully configurable (check out the OpenOCD config pages for all it can do) and makes a great debugger.

Cons

Since we are bit-banging our way to success here, it is a bit on the slow side. It’s not unusable, but it is slower than a native interface. If you don’t have one of these chips already at your disposal, I’d opt for one of the FT232H or better yet the FT2232H, both of which have an MPSSE engine which greatly improves emulation.

The post JTAG On the Cheap with the FTDI FT232R appeared first on Jacob N Calvert.