More fun with Linux and serial ports on slow hardware

This is a never ending story for me. The first time I’ve had problems with Linux’s handling of serial UART dates back to 2005 (see my previous blog post on buffer overruns). At that time I could improve the situation by applying two patches (kernel-preempt and low latency).

One year later, I have a situation where the buffer overruns are again easily reproducible at slower baudrate (54 kbauds), arguably there’s more than a serial application running this time, and it looks like the load generated by other processes (mainly watching digital I/O) renders the system less reliable with respect to its handling of serial ports.

This time I follow the advice to try out the 2.6 kernel because many “real-time improvements” (coming from the -rt branch, check its wiki) and “embedded improvements” (coming from linux-tiny) have been merged.

So I tried the stock kernel with very bad result. Results are better than the stock 2.4 but they are worse than the patched 2.4.
So I decide to try the -rt patch on 2.6, but this patch doesn’t work on my CPU card and my bugreport didn’t lead to any fix (nobody responded even though I tried hard to include the necessary information and I was ready to do whatever I would have been asked to try).

At the same time I explain my problem on the linux-kernel mailing list.
The discussion doesn’t answer my questions but still brings two ideas to try out. In the end, with two simple tweaks to the stock 2.6 kernel (mainly configuring the UART to send the interrupt as soon as the first character arrives, instead of waiting for 8 chars to accumulate in the FIFO) I have been able to get something better than the patched 2.4. And it turns out my first choices for the 2.6 kernel configuration have not been very wise so the comparison between stock 2.4 and stock 2.6 above doesn’t mean much.

Unfortunately, even if better than the patched Linux 2.4, it still doesn’t give good results in some conditions. So my primary question remains: is there a way to patch my kernel so that it will handle the serial related tasks (servicing interrupts from the UART mainly) as its primary job ? I don’t mind if such a change impact negatively the speed of the system if I can make sure that my serial exchanges are reliable.

And by reliable I mean of course no buffer overruns, but there’s a second similar problem that has been discovered: when using software handshake, the system can send out between 10 and 25 characters after the partner has sent its XOFF. Sending up to 6 characters after the XOFF is ok, more is asking for troubles because the UART on the other side will probably encounter a buffer overrun…

If you have any idea on how to resolve my problems, by any means, let me know.