QL Ethernet expansion card project

[ppe]

There was an interesting test with Q68 and the CP2200 Ethernet controller where, I think, Jan measured throughput to RAM disk to be around 35kB/s.

Q68 TFTP file transfer througput with Wolfang's driver is > 160 kBytes/s here, even with SD card.

Just to clarify and for the record so as not to offend anyone, my intention was definitely *not* to spread any kind of misinformation, I was writing from memory and should've double checked the forums on the original post. So thank you for pointing the correct figure out. I might have been a bit too excited by my results

Also, it's not a competition, Q68 will, of course, have a higher throughput. My point was merely to explore the limits of throughput on a black box when the CPU doesn't need to process TCP and is limited essentially by memory read/write speed from a HW FIFO to main memory.

TFTP vs HTTP is actually quite interesting. There's a bit of overhead in HTTP in terms of parsing the response headers. On the other hand, an HTTP client enables getting, files across the Internet (e.g. from Dilwyn's site...) directly to the BBQL. On a local network when you can configure a local TFTP server you can get rid of the header parsing which helps with very small files.

[Peter]

My code is reading from a word-wide memory mapped IO port that goes to a FIFO on the W5300 and writing to QL main memory in a two-instruction (MOVE + DBF) loop.

You should definitely think about some loop unrolling.
Also you might want to consider MOVE.L instructions, if you can decode the I/O register in a way that it appears on 4 conscutive addresses.

I might have been a bit too excited by my results

40 KBytes/s for a non (S)GC QL is an excellent result.

On the other hand, an HTTP client enables getting, files across the Internet (e.g. from Dilwyn's site...) directly to the BBQL.

That's certainly an advantage. TFTP works over the internet as well, but that is more suitable for QL-to-QL (or Q68) transfer.

[ppe]

My code is reading from a word-wide memory mapped IO port that goes to a FIFO on the W5300 and writing to QL main memory in a two-instruction (MOVE + DBF) loop.

You should definitely think about some loop unrolling.
Also you might want to consider MOVE.L instructions, if you can decode the I/O register in a way that it appears on 4 conscutive addresses.

Thank you for your comments, Peter. Loop unrolling will help, yes. I'll do a bit of experimentation with that.

Unfortunately, the FIFO interface is only two bytes wide on the chip so word-sized reads is the best that can be done with the current prototype. I guess one trick that could be used to enhance this would be to add some hardware logic between the W5300 chip's external interface and QL's bus so that the glue logic would run the W5300 side at a higher clock frequency and map the two-byte wide FIFO interface to a four-byte wide I/O memory range on the QL side. Preventing FIFO buffer underflow in the W5300 in this case would require some thought. For the time being, I'll stick to writing more software instead of tinkering with the HW. It's an intriguing possibility, nonetheless.

On the other hand, an HTTP client enables getting, files across the Internet (e.g. from Dilwyn's site...) directly to the BBQL.

That's certainly an advantage. TFTP works over the internet as well, but that is more suitable for QL-to-QL (or Q68) transfer.

Absolutely correct: when you control both sides, TFTP works fine. My point, which I failed to express clearly, was that you can download things from sites that you don't control and are only offering HTTP. TLS is a bit of a challenge - I have some thoughts on it but don't want to get distracted by it at the moment.

As an aside, one more thing I find useful about having a network card on the black box is being able to automatically set the clock on boot using Network Time Protocol client even when hardware with battery backed clock is not available.

[ppe]

You should definitely think about some loop unrolling.

Thank you for your comments, Peter. Loop unrolling will help, yes. I'll do a bit of experimentation with that.

Unrolled move.w x20 so that clock cycles per byte goes down from 21 cycles/byte to roughly 12.9 cycles/byte

Wallclock time for a 114kB file direct to memory is now approx 1.6s (non-scientific measurement...) or around 70 kB/s

[Peter]

My point, which I failed to express clearly, was that you can download things from sites that you don't control and are only offering HTTP.

Fully agreed, and I don't think you were unclear.

As an aside, one more thing I find useful about having a network card on the black box is being able to automatically set the clock on boot using Network Time Protocol client even when hardware with battery backed clock is not available.

Good point - and fortunately NTP does not require TCP, so it could be covered using the recent native UDP drivers.

Unrolled move.w x20 so that clock cycles per byte goes down from 21 cycles/byte to roughly 12.9 cycles/byte

Wallclock time for a 114kB file direct to memory is now approx 1.6s (non-scientific measurement...) or around 70 kB/s

A very good result. Also encouraging for ethernet drivers on the QL in general. It was without (S)GC?

[ppe]

Unrolled move.w x20 so that clock cycles per byte goes down from 21 cycles/byte to roughly 12.9 cycles/byte

Wallclock time for a 114kB file direct to memory is now approx 1.6s (non-scientific measurement...) or around 70 kB/s

A very good result. Also encouraging for ethernet drivers on the QL in general. It was without (S)GC?

Yes, black box QL with 640kB memory.

[ppe]

Project update, I have created an alpha version of a TCP_ device driver which supports PRINT, INKEY$, and INPUT and is enough to run QLirc by Tim and Dilwyn

[ppe]

Project update: the card and driver now work with Super Gold Card, too.

[tofro]

Project update: the card and driver now work with Super Gold Card, too.

Super extra nice, Petri!

I hope you will share the design once done?

[ppe]

Super extra nice, Petri!

Thank you for your kind words! I really appreciate it.

I hope you will share the design once done?

I'm not a production-grade electronics designer but I shall make the basic schematic idea available. There isn't much to it, really, just mapping a piece of memory to the ethernet controller's memory and dancing the QL bus dance with the control signals. Actually, in the case of W5300, one should talk about a "TCP/UDP controller" rather than an "ethernet controller".

The software part is more interesting. I don't know if any other device drivers have been written in C on the QL but it's been quite a ride. A lot of time was wasted on false starts and wrong (for me and the job) toolchains. Code archaeology - trying to decipher some of the more murkier corners of how device drivers actually work on the QL - has also taken a lot of time. Overall, it's been much more work than I ever anticipated, I was way too optimistic. "Simple" things like DNS name resolvers can be surprisingly hard when you don't have huge amounts of memory and CPU cycles to spare. You just have to think about space/time optimisations a lot more than in modern wasteful programming.

But it's also been fun most of the time. And it's a stark reminder of how different the profession of software development is in these days in terms of the things you typically consider when writing a piece of code. Provocatively it could be said that modern web development on the front and backend is pretty much just slapping various libraries together with glue.

I shall make the sources to the driver available at some point - still thinking of what the license should be. I'm hoping to have the energy to also write some blog posts about some of the gotchas I ran into on the way. Perhaps documenting some of the (for me) trickier parts may be interesting to future code archaeologists.

Well, I'm rambling again. Will be back with further updates soon