In the News: E-KERMIT ASSISTS IN HURRICANE TRACKING (September 2004)

E - K e r m i t

Version: 1.7 Date: 6 June 2011

DOWNLOAD

Contents

1. THE CONTROL PROGRAM
2. FILE TRANSFER
3. SOURCE CODE
4. THE UNIX VERSION
5. PORTING TO A NEW PLATFORM
6. DEBUGGING
7. RELEASE HISTORY
8. DOWNLOAD

EK (Embedded Kermit, E-Kermit) is an implementation of the Kermit file transfer protocol written in ANSI C and designed for embedding in devices or firmware, use in realtime applications, or for construction of DLLs and libraries. EKSW is a new version of E-Kermit that includes true sliding windows packet transport. EK and EKSW should be reconciled back into a single code base but so far that has not happened.

What E-Kermit Does

EK performs just two functions: sending files and receiving files. It is compact, portable, and fully reentrant. On SPARC (RISC), kermit.o is about 25K. On Intel (CISC) it's about 15K. By reducing buffer sizes and eliminating optional or unwanted features, smaller sizes can be achieved.

What E-Kermit Does NOT Do

EK does not include client/server functions; a command or script programming language; character-set conversion; transport encryption; or any form of communications or file input/output. It does not dial modems, it does not make connections, it does not have a built-in TCP/IP stack or interface to an external one. If you need these features, then you should use a full Kermit program, such as C-Kermit or Kermit 95.

EK is not an application itself, it's a subroutine to be called from your master application. It is useful only to developers, who must supply the master application or calling environment as well as the file and communications i/o routines. The calling environment must, in turn, make and configure the communications connection if one is required and not already open. A sample calling environment and i/o support are provided for Unix.

Customers have adapted EK to various environments and platforms, including the Palm Pilot, various kinds of technician equipment (e.g. for diagnosis and maintenance of cell phone towers), and sometimes they contribute their adaptations or i/o routines, and we can make these available on a strictly as-is basis. We are not able to support or maintain customer-contributed code; thus (for example) if a new version of EK is released, customer-contributed modules are not necessarily updated. Customer-contributed code includes:

• Microsoft Windows 9x/ME/NT/2000/XP/Vista/7 serial-port and file i/o for EK 1.3 and later.
• Wind River VxWorks for EK 1.1.
• EK 1.2 translated into Java.

EK includes the following Kermit Protocol features:

• Long packets
• Sliding windows with Go-Back-to-N error recovery (true selective repeat in EKSW).
• Repeat-count compression
• Control-character prefixing and unprefixing
• 8th-bit prefixing (for transferring 8-bit data on 7-bit links) (= parity)
• Attribute packets (type, size, and date)
• Sending and receiving single or multiple files.
• Automatic per-file text/binary mode switching.
• All three block check types (6- and 12-bit checksum, 16-bit CRC).
• Status reports (protocol state, file name, size, timestamp, bytes so far).
• Transfer cancellation by either party.

The following Kermit Protocol features are not implemented:

• Sliding windows with selective retransmission
• Character sets
• Locking shifts
• Client/server

Timeouts would be the responsibility of the Kermit program on the other end of the connection or, if needed in E-Kermit itself, the platform-dependent packet-reading routine which you would write.

As of version 1.5, E-Kermit includes preprocessor constructions to let you exclude various features such as long packets, sliding windows, and higher-order block checks to achieve the smallest possible memory footprint, and can also be built in a Receive-Only configuration.

THE CONTROL PROGRAM

EK is designed to work in a cooperative multitasking environment but does not require such an environment. The control program takes care of scheduling. Here's what the control program must (and/or can) do:

• If desired, open the communications device, if any.
• If desired, put the communications device, if any, into "packet mode".
• Initialize the kermit struct with desired operating parameters.
• Call kermit(K_INIT, ...) to have Kermit initialize itself.
• If sending files, call kermit(K_SEND) to start the transfer.

(When E-Kermit is to receive files, it waits passively for first packet from the file sender; thus it simply enters the packet loop.) In the packet loop, E-Kermit:

• Gets a buffer and reads an incoming packet into it.
• Checks for user interruption.
• Calls kermit(K_RUN, ...) to do the next step in the protocol.
• Does whatever else it wants (e.g. runs other tasks).
• Quits or continues the loop based on the kermit() return code.

Each time the control program calls the kermit() function, this grants it permission to handle one packet; thus one packet = one time slice. If the control program has nothing else to do, it simply processes packets continuously, like a regular Kermit program. While in the data-transfer loop, every kermit() call returns a struct containing:

• The current protocol state;
• The current filename;
• The file's size, if known, or -1;
• The file's timestamp, if known;
• The number of bytes transferred so far.

When done, the control program:

• Restores and (if desired) closes the communications device.

The function codes that the control program may call kermit() with are:

K_INIT   -- Initialize data structures.
K_SEND   -- (Sending only) -- Initiate sending.
K_RUN    -- Run the protocol.
K_STATUS -- Return a status report in the k_response struct.
K_QUIT   -- Quit immediately and silently.
K_ERROR  -- Send Error packet, then quit.

The return codes of the kermit() function are:

X_OK     -- OK, protocol active.
X_DONE   -- OK, protocol complete.
X_ERROR  -- Fatal error.
X_STATUS -- Returning status in response to K_STATUS.

(In fact status is retuned with every call.) Protocol state codes are:

-1       -- Fatal error
 0       -- Receiver (protocol not running)
 1       -- Receiver waiting for S packet
 2       -- Receiver waiting for F or B packet
 3       -- Receiver waiting for A or D packet
 4       -- Receiver waiting for D or Z packet
10       -- Sender (protocol not running)
11       -- Sender sent S packet (start)
12       -- Sender sent F packet (filename)
13       -- Sender sent A packet (attributes)
14       -- Sender sent D packet (data)
15       -- Sender sent Z packet (EOF)
16       -- Sender sent B packet (EOT)

FILE TRANSFER

Because EK is designed primarily for embedding, it does not use streaming or true sliding windows (although much of the sliding windows code is there). This is for the following reasons:

• Using the regular ACK/NAK protocol allows the control program to regain control after every packet. This lets it multitask, put up a graphical file-transfer display, whatever. Streaming or sliding windows could put the control program out of business for long periods of time.

• Streaming or true sliding windows would make the interface between the control program and the kermit() module much more complicated, and in fact, would push a lot of protocol details into the control program's space, where they don't belong.

• Streaming can be used only on reliable connections (such as TCP/IP) but devices with embedded communications generally use serial ports.

The lack of true sliding windows in EK is compensated by having EK pretend to support them without really doing so. This allows its sending partner to "stream" packets rather than waiting for ACKs after each one, as long as there isn't an error. If there is an error, the recovery strategy is "go back to n" (or perhaps in some cases "error out") rather than "selective repeat". EKSW, a separate program that has not been integrated with EK (but should be), supports true sliding windows with selective repeat; that is, only those packets are retransmitted that actually need to be.

In any event, since EK is intended primarily for embedding, it is anticipated that round-trip delays won't be a big factor; connections will generally be local, short, relatively fast, and if the connection is effectively flow controlled, error-free. When effective flow control is lacking, the speed and/or packet length and/or window size can be set to a combination of values that maximizes throughput and minimizes data loss.

SOURCE CODE

The source files are:

platform.h

Header file for any needed platform-specific #includes or definitions. Required, even if it's empty, because kermit.c includes it.

kermit.h

Header file for all modules. Definition of k_data and k_response structs.

kermit.c

This is the Kermit protocol engine. It is driven entirely by its call data. All state info is saved in the kermit data structure, which is passed by reference from the main module and among all the functions in the kermit module and back again to the main module; thus it should be possible for the same module to transfer multiple files at once on different connections. Furthermore, there are no library references in the kermit module, none at all, not even stdio (except when debugging is enabled), and no /usr/include/* header files are included. Rules for kermit.c:

• No global variables (except for debugging) or buffers.
• No initializing of arrays by compiler.
• Just for safety, no initializing automatic scalars either.
• No library or system calls, no #include <...>.
• All communications i/o is done by functions defined in separate modules.

The single entry point for the kermit.c module is the kermit() function:

int kermit(struct k_data * k, struct k_response * r)

The k structure contains all the operating parameters, variables, state information, and buffers; the r struct keeps the caller informed of the current state of the protocol, filename and file info, and transfer progress (bytes so far).

main.c

Sample control program. In the Unix testbed, this is just the traditional main(), which reads command-line arguments, initializes the protocol, then calls the protocol module in a state-driven loop until its work is done, then cleans up. In the embedded environment, these functions would be integrated into the control program.

unixio.c

I/O functions for Unix. Substitute your own module that implements these functions in the target environment and modify your build procedure to link with it. Entry points and calling conventions described below.

THE UNIX VERSION

Development of EK takes place on a conventional Unix platform, such as Solaris, HP-UX, or Linux, in which EK is built as a remote-mode Kermit file transfer program, similar to G-Kermit, and tested against a desktop Kermit such as K95 or C-Kermit. NOTE: The Unix version works over stdin/stdout; the "line" is conditioned in the stupidest possible way (system("stty ...")). This gives variable results; e.g. downloads from EK on Solaris might run at 17Kcps, whereas downloads from Linux on the same net to the same PC might run at 1700Kcps. This is not worth worrying about because EK is not intended for production use on Unix, which already has G-Kermit and C-Kermit for production.

The Unix makefile has the following targets (it's easy to add more):

gcc:   Build with gcc (default).
cc:    Build with cc.
hp:    Build for HP-UX.
gccnd: Build with gcc, no debugging.
gprof: Build with gcc, include profiling.
clean: Remove object and core files.

The makefile creates a Unix executable called "ek" (embedded kermit). The sample main() routine provides a simple command-line interface:

$ ./ek -h
Usage: ./ek options
Options:
 -r           Receive files
 -s files     Send files
 -p [neoms]   Parity: none, even, odd, mark, space
 -b [123]     Block check type: 1, 2, or 3 (default = 3)
 -k           Keep incompletely received files
 -B           Force binary mode
 -T           Force text mode
 -R           Remote mode (vs local)
 -L           Local mode (vs remote)
 -E number    Simulated error rate (0-100)
 -d           Create debug.log
 -h           Help (this message)
$

When sending files, if you don't specify Text or Binary, EK scans each file and chooses text or binary mode based on its contents.

Remote vs Local mode is used only to enable the test for keyboard interruption of file transfer.

PORTING TO A NEW PLATFORM

Version 1.0 of EK was ported to VxWorks by Airvana, Inc, Chelmsford MA. The complete VxWorks EK 1.1 package is included as an example of a production system by Airvana's permission (note that the EK API has changed slightly since then, so before the VxWorks code can be used, it must be updated). To port to a new platform:

• Add a new Makefile entry for your target, or write your own build procedure.

• Create a platform.h file for your platform. This can include any desired #include's or definitions, and it can also be used to override certain definitions in kermit.h:

  #define NODEBUG to build without debugging code.
  #define HAVE_UCHAR if UCHAR is already defined or typedef'd to unsigned char.
  #define HAVE_ULONG if ULONG is already defined or typedef'd to unsigned long.
  #define IBUFLEN to be the desired size for the file input buffer.
  #define OBUFLEN to be the desired size for the file output buffer.
  #define FN_MAX to be the maximum length for a filename.
  #define P_PKTLEN to override the default maximum packet length.
  #define P_WSLOTS to override the default maximum window slots.

• Replace the sample main.c with your own control program. Use the same header files and calling conventions as in the sample.

• Copy unixio.c to xxxio.c (name of your choice), edit it to work on your platform using exactly the same calling conventions, and adjust your build procedure to link with your new xxxio module instead of unixio. Note that the fill input and output buffers (i_buf[] and o_buf[]) must be defined in your xxxio routine.

Here are a few tips for creating an i/o module:

The device i/o routines are expected to handle communications parameters themselves, including communication line speed, parity, and flow control. In particular, Kermit does not handle parity, but still must be told about it. This is done in the setup by main(). Your readpkt() and tx_data() routines should strip and add parity, respectively, if necessary. On serial connections, maybe the UART can be programmed to do this.

API Change between EK 1.1 and 1.2: Calling conventions (function argument lists and return values) were changed between version 1.1 to 1.2, mainly to give all routines access to the k struct in a consistent way, and also to provide better feedback to the caller. In each case where a change was made, both the old and the new format are shown.

The device i/o functions are:

int
devopen(char * device)

Opens the given communications device. Might also be a network host, whatever. Returns 0 on failure, 1 on success.

int
devsettings(char * settings)

This one performs any needed settings for the device, such as speed and flow control for a serial device. Since there's no way to know what the relevant parameters are, this routine just takes a string, which can be in any format, e.g. "9600;8N1" or "speed=57600;flow=rts/cts"; the devsettings routine will have to parse the string. Returns 0 on failure, 1 on success.

int
devrestore(void)

If desired, put device back the way devsettings() found it, e.g. just before closing it.

int
devclose(void)

Closes the communications device.

int
readpkt(UCHAR * buffer, struct k_data * k) (1.1)

readpkt(struct k_data * k, UCHAR * buffer, int length) (1.2)

This routine must do exactly what the sample one does: search for the start of packet, then copy all the characters up to (but not including) the end of packet into the packet buffer whose address is given. You'll want to code this as efficiently as possible, using whatever tricks are available to you: nonblocking buffered reads, etc. If you want your Kermit program to time out, this is where you would put the code. NOTE: Timeouts are not necessary, since the chances that ek's Kermit partner can not time out are about 0. The EK 1.2 format puts k as the first argument for consistency with other routines, and adds a buffer length argument.

Note the F_CTRLC feature. This is enabled by default. It allows EK to be broken out of packet mode by sending it three consecutive Ctrl-C's in the data stream. You normally would not need to disable this since, even if the sender is "unprefixing" Ctrl-C, three of them in a row would normally be collapsed into a repeat-count sequence.

int
tx_data(UCHAR * data, int length, short parity) (1.1)

tx_data(struct k_data * k, UCHAR * data, int length) (1.2)

Here again, you must tack on parity (if it is not being done automatically by the communication device or driver). This routine should be both efficient and robust. It is supposed to transmit the entire data string or else fail. See the unixio.c sample for what I mean by "robust". In EK 1.2 and later, the parity setting is picked up from the k struct.

The file i/o functions are as follows; of course they can be used for reading or writing anything -- not just files: memory, tape, cards, laser beams, instrument controllers, whatever. It doesn't matter what you call these routines, but the argument list and return type must be as shown; also if you give them different names, you'll have to change the prototypes in kermit.h:

int
openfile(UCHAR * filename, int mode, struct k_date * k) (1.1)

openfile(struct k_date * k, UCHAR * filename, int mode) (1.2)

Opens the named file in the given mode (1 = read, 2 = write, 3 = append). Returns X_OK on success, X_ERROR on failure.

ULONG
fileinfo(UCHAR * filename, UCHAR * buf, int buflen, short * type, short mode) (1.1)

fileinfo(struct k_data * k,UCHAR * filename,UCHAR * buf,int buflen,short * type,short mode) (1.2)

Gets information about the specified existing local file: size, date, and, if mode == 0, the file type (text or binary). buf and buflen apply to the file's date/time string. Returns X_OK or X_ERROR.

int
readfile(struct k_data *)

Reads a buffer from the input file, and if the transfer is in text mode, converts the record format to standard Kermit Stream CRLF. Returns X_OK or X_ERROR.

int
writefile(struct k_data *, CHAR * buffer, int length)

Writes a buffer to the output file, and if the transfer is in text mode, also converts the standard Kermit Stream CRLF record format to whatever is required locally. Returns X_OK or X_ERROR.

int
closefile(struct k_data *, UCHAR code, int mode)

Closes the file. For output files, of course this flushes any pending buffers to the file before closing it; then it checks to see if the sending Kermit canceled the file transfer before it was finished (code == 'D'), in which case it discards the partial file instead of keeping it. The mode indicates whether it's an input or output file, so incompletely received files can be deleted if desired. Returns X_OK or X_ERROR.

The precise calling conventions are shown in the unixio.c file.

DEBUGGING

If EK was built without NODEBUG defined, then if you include the -d option on the command line, the Unix-based sample version of EK creates a debug.log file in its current directory. In the production version, you would add -DNODEBUG to the C compiler CFLAGS to eliminate the debugging code. The sizes shown above include debugging. You can implement the debug function any way you want in your platform-specific i/o module.

RELEASE HISTORY

Version	Date	Description
1.1	2002/10/07	Initial release. VxWorks version still at this level.
1.2	2003/01/28	Improved API, Java port (which is still at this level).
1.3	2004/03/04	Fix file transfer with HyperTerminal.
1.4	2004/03/20	Fix reception of empty files.
1.5	2004/04/10	Fix problem with A-packets, allow super-small and/or receive-only configurations.
1.51	2004/09/23	Adapt to Philips XAG30 (John Dunlap)
EKSW 0.94	2010/06/24	True sliding windows with selective retransmission (John Dunlap)
1.6	2011/03/30	Published and released under the 3-Clause Revised BSD License.
1.7	2011/06/06	FORCE-3 protocol, works together with C-Kermit 9.0 Beta.01 (explained here)

DOWNLOAD

Several different E-Kermit implementations are available for download. E-Kermit itself, version 1.7, is the primary one. The others are adaptations to different platforms or languages that were done during prior releases of E-Kermit, as indicated in the previous section; in other words, the fixes found in E-Kermit 1.3, 1.4, and 1.5 are not in the VxWorks or Java versions, and the VxWorks version uses the E-Kermit 1.1 API instead of the improved 1.2 version. EKSW has some modifications to the API and other inconsistencies that should be undone before it can be integrated with EK 1.6, but is perfectly usable on its own. In fact, this is the version running in the new generation of Apex-EM ocean floats and has been tested more thoroughly under more adverse conditions than possibly any other Kermit protocol implementation. Giving rise to version 1.7, which implements the new Force-3 error-checking packet protocol. (EKSW will get this too during the summer.)

		HTTP		FTP
Name	Description	Tar*	Zip	Source Files
E-Kermit 1.7	Portable to all platforms, with Unix demo.	Download	Download	Download
EKSW 0.94	E-Kermit with true sliding windows, adapted to Linux.	Download	Download	Download
EKVX 1.1	E-Kermit 1.1 adapted to VxWorks.	Download	Download	Download
Java	E-Kermit 1.2 converted to Java	Download	Download	Download
Simirid	Protocol stress tester [description]	Download	Download	Download

* Not compressed, no need, they are very small.

Click here for FTP access to all E-Kermit files.

[ Top ] [ Kermit Home ]

---

Embedded Kermit / The Kermit Project / Columbia University / kermit@columbia.edu / 17 June 2011