Imaging floppy disks with the DiscFerret and Magpie

The DiscFerret is a USB-based platform for archiving floppy and harddisks. It consists of both hardware and software. Magpie is the software that reads and dumps complete tracks in a file. Both the DiscFerret board and the Magpie program are produced by Philip Pemberton.

This is not meant to be a blog, just an unorganised listing of snippets of information which I would have forgotten next time I work with the DiscFerret.

My main goal is to produce images in formats used by emulators. This means writing conversion software to other imaging formats like DMK and ImageDisk. To get an idea of what one could do with a DiscFerret, I tried to make a classification. My usage is clearly the first category, "Archiving of floppy disk images".

• The Discferret is not USB powered. Supply voltage is 12 V DC, pin positive, barrel negative. The 3V3 LED should light up. Connect USB before power. - The libdiscferret installes dynamic libraries in /usr/local/lib which is not used by Ubuntu. I solved it by: sudo ln -s /usr/local/lib/libdiscferret.so /usr/lib before reading the DiskFerret:Downloads page which says: To change this, append "PREFIX=path" to both the "make" and "make install" commands.
• When actual reading, the MCU and FPGA LEDS are lit alternately.
• sudo ./magpie --verbose --drive pc35a --format FAT16 --outfile testFile The 'pc35a', being a PC 3.5" disk drive, comes from the ./script/drive/*.lua files.
• flatcable connector is not 34 p. but 40 pins. The lower 34 pins are for floppy drives (from Philip Pemberton).
• magpie produces files in DFI image format (http://www.discferret.com/wiki/DFI_image_format, new format)
• The DFI image format seems to indicate periods between flux transitions.
• Typical run of magpie:

  $ sudo ./magpie --verbose --drive pc35a --outfile ibmdos330.dfi
  Verbose mode ON
  Loading microcode...
  Microcode loaded successfully.
  Connected to DiscFerret with serial number GB0L0801
  Revision info: hardware 0I06, firmware 28
  Microcode type 56661, revision 40
  
  Drive type: 'pc35a' (PC 3.5", twisted cable (CBL-01A), drive A)
  135 tpi, 84 tracks, 2 heads.
  Recalibration attempt 1 succeeded.
  Measured disc rotation speed: 302.389 RPM
  Acquiring data from disc at 100MHz
  CHS 0:0:1, 182498 bytes of acq data
  CHS 0:1:1, 188670 bytes of acq data
  CHS 1:0:1, 190711 bytes of acq data
  .... (more tracks)
  CHS 82:1:1, 162158 bytes of acq data
  CHS 83:0:1, 162727 bytes of acq data
  CHS 83:1:1, 162126 bytes of acq data
  Moving heads back to track zero...
  Recalibration attempt 1 succeeded.
  $ ls -l ibmdos330.dfi
  -rw-r--r-- 1 root root 30396820 2012-01-15 10:20 ibmdos330.dfi
  

• Re-phrasing the DFI image format as produced by magpie:

  0  DFE2  - 4 byte header
    [
  3  trackHi - track high byte
  4  trackLo - track low byte
  5  headHi  - head high byte
  6  headLo  - head low byte
  7  sectHi  - sector high byte
  8  sectLo  - sector low byte
  9  nbyte3  - number of bytes high byte
  A  nbyte2  - number of bytes mid-high byte
  B  nbyte1  - number of bytes mid-low byte
  C  nbyte0  - number of bytes low byte
    ] per track
  

  For the partial example file (720kByte/double sided/double density) this results in this data interpretation:

  44 46 45 32  - header
  00 00        - track 0
  00 00        - head 0
  00 01        - sector 1 (used for hard sectored floppies only)
  00 02 C8 E2  - number of bytes read
  .. .. .. ..  - 0x2C9E2 bytes
  00 00        - track 0
  00 01        - head 1
  00 01        - sector 1 (used for hard sectored floppies only)
  00 02 E0 FE  - number of bytes read
  .. .. .. ..  - 0x2E0FE bytes
  00 01        - track 1
  

• Viewed with a hex-editor, the first part looks like this:
  
  This file is created with the default click frequency of 100MHz. All "7F" values are counter overflows, and should be added to the following non-7F value. The 25MHz setting has enough resolution for floppy usage, and produces considerable smaller files.
• It seems a bit silly to add comment to pseudo code, but it helped me:

  carry = 0
  For every byte in the stream:
    if ((byte AND 0x7f) == 0x7f):    // 0x7F = counter overflow, add this to total
      carry = carry + 127
    else if (byte AND 0x80) != 0:    // 0x80 = index detect 
      carry = carry + (byte & 0x7F)  // add lower 7 bit value to carry
      add_index_position(carry)      // add high bit to carry
    else:                            // here byte < 0x7f
      emit((byte AND 0x7f) + carry)  // not over flow or index; flux transition
      carry = 0                      // restart counter; reset carry
  if carry > 0:                      // 
    emit(carry)                      // report carry; either 0x7F or 0x80
  
  // emit() looks like a written value
  // add_index_position() is just setting the high bit of the last data value 
  // 
  

• It looks like the DFI file contains track headers and time intervals between changes in magnetic polarity. Each non 7F value indicates such an transition. The time interval is the sum of the value and all preceding 7Fs, probably in 100MHz ticks. One has to analyse or know the format (FM, MFM, ...) to interpret the values into DAMs and data.

Typical launch of magpie:

sudo ./magpie --verbose --drive pc35b --outfile <fileName>.dfi -c 25

remarks, not important enough to be an issue:

• The magpie usage text doesn't report all relevant options
• The magpie drivetype values are not listed in the usage. But they come from the script/drives/*.lua files.
• http://www.discferret.com/wiki/DFI_image_format states sector number is zero for soft-sectored floppies. It is '00 01'.
• The current version does not have a clear version number or a way to report it.
• Given the functionality of supporting multiple DiscFerrets, an option to list all found DiscFerrets, with the id format used by the -s option would be useful. This might be automatic if multiple DiscFerrets are connected, but I cannot test this. The source 'suggests' the first is selected silently.
• The current 'DFE2' format does not contain information on the sample rate.
• As there is no indication of the sample rate in the .dfi file, how does an analysing program know how to interpret the data? Merlin apparantly does, as both 25 MHz and 100 MHz files appear to have ~81 ms as minimum flux transition distance.

h

usage message

d

drivetype. Standard these are supported: pc35a, pc35b, amstrad_eme23x_a, amstrad_eme23x_b, vfo_8inch_0, vfo_8inch_1, vfo_8inch_2, vfo_8inch_3. Configuration is in the scripts in scripts/drive/*.lua.

f

formattype. Optional. Standard these are supported: gen40, gen80ds. Configuration is in the scripts in scripts/format/*/lua

s

serial number. Optional, only needed if more than one discFerret is connected.

o

output file. Mandatory. The output format is always .dfi ('DFE2', v1.1)

c

clock frequency. Optional. Can be 25, 50 or 100 MHz. Default is 100 MHz, for normal floppy usage 25 MHz is enough.

Data interpretation

Based on the cw2dmk code I created a tool to visualize the data in a dfi image.

• version 0.1. This is just a proof of concept, hardcoded
• version 0.2. This version still doesn't produce useful output, but can correctly read FM and MFM floppies. Autodetection not operational yet.
• version 0.3. This version produces JV1 output. Adding more formats is easy, but not a priority.
• version 0.4. This version has better defined log levels and a better internal structure. And I found out the dumped jv1 file actually worked :-).

Disk formats:

• DSK formats Jeff Vavasour's JV1 & JV3
• DMK format David Keil's DMK
• D88 format Panasonic Pasiopeia D88
• IMD format Dave Dunfields Image Disk

Last updated: 2012-04-14

e-mail