Brewing beer is fun, but like most hobbies, once you dip your toes in, you find that more equipment results in more fun and better results. In the case of brewing, temperature control is one of the most fundamental aspects.
I've been using a Grainfather for mash and boil steps and a hacked freezer with an Inkbird controller for fermentation and cold crash. For serving beer, I have a Keg King series 4 kegerator with Intertap taps.
Lately I've been looking at improving the fermentation stages. Ideally, I'd like to build a closed system capable of temperature controlled fermentation under pressure. While I am not too impressed with the Grainfather, iMake (the company that makes the Grainfather) have other products that seem interesting. At the moment, they are running a competition where you get to build your own wish list, so I picked a few items that might come in handy when it comes to mash, boil and fermentation.
25 November 2017
26 November 2014
All I want for...
All I wanted for my birthday was a pair of Sharks Pants and a Black Albert? Was that too much to ask? Apparently!
Luckily Leura Cellars had some pretty awesome consolation prizes and they delivered fast!
Luckily Leura Cellars had some pretty awesome consolation prizes and they delivered fast!
04 August 2011
SDHC Card performance
I've been recording a few 1080p videos with a Panasonic Lumix GH1 camera, using a Transcend 32 GB Class 6 card. Lately, the camera started reporting errors, complaining about the card being too slow. I also experienced a few write errors on the camera, which resulted in the camera reporting that the card is faulty. Formatting the card did not help. To make sure that the camera is not at fault, I tested with a Sandisk Class 4 card. As expected, there were no issues with the Sandisk card and it was fast enough to record video.
I decided to perform some more thorough testing on the Transcend card using a Linux system. I decided to examine the sequential write performance of the entire card. For this, I started KDE System Monitor and configured it to plot the read and write speeds on the device. I then used dd to write zeros to the entire card, using sequential writes in 4MB blocks. I also repeated the test with 4kB blocks. Given that this type of test bypasses any filesystem fragmentation issues, a Class 6 card must never write slower than 6MB/s - on any part of the media. As you can see from the attached screen shots, the card performed to specification only in certain areas. There are a number of occasions where the write performance is well below the requirement of a Classs 6 device.
Interestingly enough, the areas of poor performance do not appear in exactly the same locations between tests and also do not seem to be of the same size. I have no definitive explanation for this, but I guess it could be due to something like a wear levelling algorithm being applied, or perhaps bad block re-mapping. Then again, it could be some other issue that's more intermittent and the slowdown is a result of multiple retries and error detection and correction algorithms doing their work. I guess the manufacturer will have means of pinpointing the issue more accurately, but it's unlikely that they would report the root cause back to the end users. Still, it would be educational to find out what goes wrong in memory cards.
Transcend offer a Lifetime Warranty on their cards, so I will contact their technical support to request a replacement. I've tried to use their on-line RMA form, but it seems that this is only available to customers in USA. We'll see how well Transcend looks after their Australian customers - I'm about to contact their technical support in Taiwan.
I decided to perform some more thorough testing on the Transcend card using a Linux system. I decided to examine the sequential write performance of the entire card. For this, I started KDE System Monitor and configured it to plot the read and write speeds on the device. I then used dd to write zeros to the entire card, using sequential writes in 4MB blocks. I also repeated the test with 4kB blocks. Given that this type of test bypasses any filesystem fragmentation issues, a Class 6 card must never write slower than 6MB/s - on any part of the media. As you can see from the attached screen shots, the card performed to specification only in certain areas. There are a number of occasions where the write performance is well below the requirement of a Classs 6 device.
Interestingly enough, the areas of poor performance do not appear in exactly the same locations between tests and also do not seem to be of the same size. I have no definitive explanation for this, but I guess it could be due to something like a wear levelling algorithm being applied, or perhaps bad block re-mapping. Then again, it could be some other issue that's more intermittent and the slowdown is a result of multiple retries and error detection and correction algorithms doing their work. I guess the manufacturer will have means of pinpointing the issue more accurately, but it's unlikely that they would report the root cause back to the end users. Still, it would be educational to find out what goes wrong in memory cards.
Transcend offer a Lifetime Warranty on their cards, so I will contact their technical support to request a replacement. I've tried to use their on-line RMA form, but it seems that this is only available to customers in USA. We'll see how well Transcend looks after their Australian customers - I'm about to contact their technical support in Taiwan.
 |
| Transcend 32GB Class 6 SDHC write performance - sequential 4MB writes. |
 |
| Transcend 32GB Class 6 SDHC write performance - sequential 4kB writes. |
15 September 2009
puppy is an old dog now
I have long ago upgraded from a Topfield PVR to a Beyonwiz PVR, so puppy has not really been getting any attention for a few years. It's an old dog now and I won't be teaching it any new tricks.
Since puppy was developed, Linux USB support has improved tremendously. As the Linux kernel matured, advances were made in the APIs that the kernel presents to user space. The supporting user space libraries have also evolved in the last five years. Current Linux kernel supports USB3 - well before Windows or OSX. Such improvements have lead to some APIs, such as usbfs, being deprecated.
The main objective behind developing puppy was to create a utility that could be used in embedded Linux devices. Almost all such embedded devices still run older Linux kernels, so I will not be upgrading the puppy code to switch from the now deprecated usbfs implementation to the new sysfs API. I don't really have the means to test the changes. Current Linux kernels still have support for usbfs, you just need to turn it on.
To bring back the
That's all you should need to make your system compatible with puppy.
Since puppy was developed, Linux USB support has improved tremendously. As the Linux kernel matured, advances were made in the APIs that the kernel presents to user space. The supporting user space libraries have also evolved in the last five years. Current Linux kernel supports USB3 - well before Windows or OSX. Such improvements have lead to some APIs, such as usbfs, being deprecated.
The main objective behind developing puppy was to create a utility that could be used in embedded Linux devices. Almost all such embedded devices still run older Linux kernels, so I will not be upgrading the puppy code to switch from the now deprecated usbfs implementation to the new sysfs API. I don't really have the means to test the changes. Current Linux kernels still have support for usbfs, you just need to turn it on.
To bring back the
/proc/bus/usb/ entries, rebuild your kernel with CONFIG_USB_DEVICEFS enabled andmount -t usbfs none /proc/bus/usbThat's all you should need to make your system compatible with puppy.
A third party patch to add sysfs support to puppy is available here. There is no matching patch for ftpd-topfield, however the above recipe should get ftpd-topfield working again.
10 April 2008
puppy 1.14 released
A new release to implement a few requests from users. In particular, support for Ubuntu and for the USB accelerator firmware patch.
Since I no longer use a Topfield PVR, the changes in this release have not been tested. The changes are fairly simple, so I don't expect problems.
Since I no longer use a Topfield PVR, the changes in this release have not been tested. The changes are fairly simple, so I don't expect problems.
10 August 2007
Artbox becomes Final Cut Server
As I posted in December last year, Apple have bought Proximity. The acquisition included a product named Artbox, which is one of the projects I worked on at Proximity. I have written most of the user interface for Artbox.
Apple are about to start shipping a new product called Final Cut Server, which is based on Artbox. There have been many changes under the hood, including a move to Apple's Compressor for transcoding, however the user interface appears to be substantially the same.
Apple are about to start shipping a new product called Final Cut Server, which is based on Artbox. There have been many changes under the hood, including a move to Apple's Compressor for transcoding, however the user interface appears to be substantially the same.
06 December 2006
Proximity technology acquired by Apple.
A short blurb on the Proximity web page says that "all Proximity technology and intellectual property was recently acquired by Apple." I used to work for Proximity until the company started experiencing cash flow issues. I certainly hope that the Apple involvement is a way to inject resources into the excellent work that was being done at Proximity. The development team we had at Proximity here in Sydney was a very talented bunch and work was very enjoyable. Proximity was the first (and possibly remains the only) Australian company to win an Emmy award.
This is not the first time that something like this has happened. Back in the 1990's I worked on embedding video streaming technology and building two way interactive systems, including VOD. The company that was originally involved in the R&D of these technologies run out of money (due to law suits on an unrelated project) and the technology ended up being acquired through a string of takeovers by Real Networks. These days, the technology I developed for embedding video content is everywhere. Unlike the guys at You Tube, I didn't get filthy rich.
Perhaps I should have patented all the work I have done - even 0.1% of a few hundred million dollars here and there would come in handy!
20 November 2006
The avatar
I've been using the same avatar for many years now. A few times people have asked about it's origin. The avatar is a cropped part of the picture on the right.
The monkey is a snow macaque in an onsen at Jigokudani, near Nagano, Japan. For those of you familiar with the movie Baraka, these are the same group of monkeys that are depicted at the beginning of Baraka, dozing in the hot spring bath.
I took the picture in 1999, while living in Japan.
15 November 2006
Long time between updates.
It's been almost a year since the last update to this blog. A lot of stuff has happened. I've done research and development work for IceTV, working on projects such as PIMP, PIMP for mobile devices, IceBox2, IceGuide solutions for a number of third party PVRs and many other projects that remain either confidential or unnamed.
As a result of the Channel Nine legal challenge, the IceTV IPO was canceled, which affected the amount of funds available for research and development. The net result was that even though IceTV would have been very keen to have my services and I would have liked to do R&D for IceTV, I had to look elsewhere to find paid work.
That paid work is being the technical lead on a new class of a home entertainment product. It's an interesting blend of new and old technology applied in a pragmatic manner. Rather than being carried away by the technological possibilities, the development has concentrated on the expectations of the average person.
There are both good and bad sides to developing such a device. The hardest part is the dumbing down of the product. There are so many possibilities that the tech-savvy power user could and would have in the device, but in the grand scheme, these features would only confuse 90% of the potential users. On the opposite side of the spectrum is implementing the obvious features, which more often than not present problems that are hard to solve. Take, for example, the ability to treat analogue and digital inputs in exactly the same way. A reasonable expectation, yet there are many scenarios where it starts getting complicated and expensive in terms of hardware design.
The product is not something I would have designed for myself, it is something intended for the in-laws and their friends. They don't care whether the phone uses CDMA or GPRS, whether the TV is analogue or digital, whether the car stereo plays MP3 or 8-track. They just want it to work without having to think very hard about which button to press. There are millions of people like that out there.
On the plus side, if we get this right, just about everyone in Australia would have heard of the product by this time next year.
As a result of the Channel Nine legal challenge, the IceTV IPO was canceled, which affected the amount of funds available for research and development. The net result was that even though IceTV would have been very keen to have my services and I would have liked to do R&D for IceTV, I had to look elsewhere to find paid work.
That paid work is being the technical lead on a new class of a home entertainment product. It's an interesting blend of new and old technology applied in a pragmatic manner. Rather than being carried away by the technological possibilities, the development has concentrated on the expectations of the average person.
There are both good and bad sides to developing such a device. The hardest part is the dumbing down of the product. There are so many possibilities that the tech-savvy power user could and would have in the device, but in the grand scheme, these features would only confuse 90% of the potential users. On the opposite side of the spectrum is implementing the obvious features, which more often than not present problems that are hard to solve. Take, for example, the ability to treat analogue and digital inputs in exactly the same way. A reasonable expectation, yet there are many scenarios where it starts getting complicated and expensive in terms of hardware design.
The product is not something I would have designed for myself, it is something intended for the in-laws and their friends. They don't care whether the phone uses CDMA or GPRS, whether the TV is analogue or digital, whether the car stereo plays MP3 or 8-track. They just want it to work without having to think very hard about which button to press. There are millions of people like that out there.
On the plus side, if we get this right, just about everyone in Australia would have heard of the product by this time next year.
24 November 2005
Thank you to those who donated.
I would like to thank all the users who donated funds after reading my post on the Toppy.org.uk forums. Some of the work I have done for the Toppy community:
You can use the Paypal button to send a donation if you have not had the opportunity to donate yet.
Your contributions are very much appreciated. The total amount donated in one week since the post is roughly equivalent to the total amount donated in the previous 12 months.
It is good to see that there are people in the community who appreciate the efforts to further evolve the limited functionality provided by vendors of various devices to a level that actually suits consumers.
The collected funds come just in time for my birthday and will be pooled together with my "birthday money" to purchase resources required for development of solutions that allow the Toppy to better integrate into your digital lifestyles.
I expect to be involved in a project that builds on top of technologies such as puppy and provides the ability to remotely control your PVR. Appropriate announcements will be made in due time.
- puppy - Linux software for talking with Topfield PVRs. Forms a basis for various other solutions, such as ftpd-topfield.
- NSLU2 firmware development - ensuring that the alternative firmwares developed by the NSLU2-Linux community can be used to connect to Topfield PVRs.
- The TAP Project at BerliOS - resources for TAP developers.
- TGD specification - the defacto standard for getting EPG and timer data onto a Topfield PVR. Cornerstone technology for both applications such as TED/TEDS and for TAPs, such as EPG_uploader.
- Open Sourcing of icebox - development and subsequent Open Source release of a commercial platform for delivering an EPG feed to Topfield PVRs. This technology uses a wireless router, customised to deliver EPG and TAPs to your Toppy without the need to have a PC connected or turned on.
- The original JavaXMLTV to TGD harvester - only a prototype to demonstrate the process of EPG delivery to the Toppy, this hack has now become a defacto standard for getting Australian EPG onto the Toppy.
(BTW: I do not endorse the use of this solution anymore - an IceGuide subscription from IceTV is preferable)
You can use the Paypal button to send a donation if you have not had the opportunity to donate yet.
Your contributions are very much appreciated. The total amount donated in one week since the post is roughly equivalent to the total amount donated in the previous 12 months.
It is good to see that there are people in the community who appreciate the efforts to further evolve the limited functionality provided by vendors of various devices to a level that actually suits consumers.
The collected funds come just in time for my birthday and will be pooled together with my "birthday money" to purchase resources required for development of solutions that allow the Toppy to better integrate into your digital lifestyles.
I expect to be involved in a project that builds on top of technologies such as puppy and provides the ability to remotely control your PVR. Appropriate announcements will be made in due time.
30 September 2005
Support for 64 bit platforms
puppy has received a few fixes to enable correct operation on 64 bit platforms. Success has been reported on amd64.
Version 1.12 of puppy will include these fixes when it is released. In the meantime, the CVS version should work for those who don't want to wait for the next release.
Version 1.12 of puppy will include these fixes when it is released. In the meantime, the CVS version should work for those who don't want to wait for the next release.
27 August 2005
Version 1.11 released.
First of all, I'd like to thank all three individuals for contributing funds towards the project. The amount of money collected from donations is minimal. Not quite enough to pay for 50% of an NSLU2. Donations are of course optional, but encouraged. Any amount will be accepted.
Thanks!
Now, onto the main news...
I have finally managed to implement workarounds for bugs in the Topfield USB implementation. Puppy 1.11 is the result of several months worth of troubleshooting, reverse engineering, USB bus snooping and other tedious work. It may have not been this difficult if Topfield had cooperated, however I could not get them to even acknowledge my emails or return phone calls.
This release has undergone rigorous testing, including the transfer of several hundred gigabytes of data and transfers of over 4,000 files in both directions. Most of the testing was performed on a Linux workstation running 2.6.12-gentoo-r7 and ehci_hcd. The tests involved uploading the data to a Topfield TF5000PVRt and then downloading them again. To verify data integrity, the md5sums of the original files and the downloaded files were compared. Some testing was also done on an NSLU2 running OpenSlug 2.5 beta.
This release changes the way turbo transfers are handled. Previous versions allowed the -t switch to enable turbo transfers only during put and get commands. The new version removes the -t switch and introduces the turbo command. The turbo command takes either 0 (zero) or 1 (one) as the argument to disable or enable turbo mode. Once turbo is enabled, it will persist until explicitly turned off.
The USB timeout has also been increased so that puppy does not timeout when the hard disk needs to spin up. While this is sufficient in most instances, there are still occassions where the timeout is not long enough. For example, deleting a directory with 2,000 files can take almost a minute. This will cause a timeout.
Thanks!
Now, onto the main news...
I have finally managed to implement workarounds for bugs in the Topfield USB implementation. Puppy 1.11 is the result of several months worth of troubleshooting, reverse engineering, USB bus snooping and other tedious work. It may have not been this difficult if Topfield had cooperated, however I could not get them to even acknowledge my emails or return phone calls.
This release has undergone rigorous testing, including the transfer of several hundred gigabytes of data and transfers of over 4,000 files in both directions. Most of the testing was performed on a Linux workstation running 2.6.12-gentoo-r7 and ehci_hcd. The tests involved uploading the data to a Topfield TF5000PVRt and then downloading them again. To verify data integrity, the md5sums of the original files and the downloaded files were compared. Some testing was also done on an NSLU2 running OpenSlug 2.5 beta.
This release changes the way turbo transfers are handled. Previous versions allowed the -t switch to enable turbo transfers only during put and get commands. The new version removes the -t switch and introduces the turbo command. The turbo command takes either 0 (zero) or 1 (one) as the argument to disable or enable turbo mode. Once turbo is enabled, it will persist until explicitly turned off.
The USB timeout has also been increased so that puppy does not timeout when the hard disk needs to spin up. While this is sufficient in most instances, there are still occassions where the timeout is not long enough. For example, deleting a directory with 2,000 files can take almost a minute. This will cause a timeout.
11 March 2005
Donations accepted, development continues.
You will find a donations button on the left hand side of this page. I will gladly accept any amount of money you care to contribute towards this project. Development is not only time consuming, but the cost of reference books and hardware can be quite high too.
Solutions like puppy ultimately benefit the vendors of the hardware by providing free publicity and increasing the sales of their hardware. Requests to Topfield for donation or loan of development hardware go unanswered. I am now asking the community to pitch in and donate towards the cause. This is entirely optional and the donation does not have to be large.
The development of puppy has not halted. The following items are on my list of things to do next:
So, send some money or even just a message of encouragement - contact details are just to the left.
Solutions like puppy ultimately benefit the vendors of the hardware by providing free publicity and increasing the sales of their hardware. Requests to Topfield for donation or loan of development hardware go unanswered. I am now asking the community to pitch in and donate towards the cause. This is entirely optional and the donation does not have to be large.
The development of puppy has not halted. The following items are on my list of things to do next:
- Make puppy work with kernel 2.6 based Linux distributions.
- Diskless version - no need for external disk or USB flash, transfer directly to a networked PC.
- Wireless version - no cables to worry about.
- Web interface - configure everything using your browser.
- Standalone flash image - just flash and use, no need to perform any complex installation steps.
So, send some money or even just a message of encouragement - contact details are just to the left.
28 January 2005
puppy wiki updated
I spent a bit of time updating the puppy wiki at http://puppy.sf.net
New users and particularly users with limited or no Linux experience, should now be able to follow fairly detailed steps, showing them how to achieve a number of tasks. These tasks range from simple things, such as changing security permissions on files, to enabling large file support, mounting networked drives, enabling secure shell and even simple scripting for doing batch uploads of .tgd files containing EPG data.
Even if you have been using the NSLU2 with puppy for a while now, I would suggest having a look at the wiki, just in case there is some new information.
As always, user contributions are welcome and encouraged. Just click the edit button present at the top/bottom of each page and add your bit. If unsure, feel free to discuss the matter at the Topfield Australia Forums in this thread.
New users and particularly users with limited or no Linux experience, should now be able to follow fairly detailed steps, showing them how to achieve a number of tasks. These tasks range from simple things, such as changing security permissions on files, to enabling large file support, mounting networked drives, enabling secure shell and even simple scripting for doing batch uploads of .tgd files containing EPG data.
Even if you have been using the NSLU2 with puppy for a while now, I would suggest having a look at the wiki, just in case there is some new information.
As always, user contributions are welcome and encouraged. Just click the edit button present at the top/bottom of each page and add your bit. If unsure, feel free to discuss the matter at the Topfield Australia Forums in this thread.
15 January 2005
Performance tuning.
I spent a bit of time on optimising puppy to improve performance on the NSLU2. The main issue with the Topfield USB protocol is that it is very sensitive to latency.
I adopted an aggressive optimisation strategy for the main codepath used in the get operation. The resulting changes improve the peak performance from 9.54 Mbits/s to a reasonably good 16.39 Mbits/s. That's approximately a 170% speed improvement and brings the performance of puppy on NSLU2 within the ballpark of Altair on a PC.
I will need to do more testing before unleashing a new version on the public, but the initial results look promising.
I adopted an aggressive optimisation strategy for the main codepath used in the get operation. The resulting changes improve the peak performance from 9.54 Mbits/s to a reasonably good 16.39 Mbits/s. That's approximately a 170% speed improvement and brings the performance of puppy on NSLU2 within the ballpark of Altair on a PC.
I will need to do more testing before unleashing a new version on the public, but the initial results look promising.
05 January 2005
puppy 1.5
A new release of puppy made it out last night. Version 1.5 adds the ability to create directories on the NSLU2. This will help in the long run as part of a fully automated EPG transfer solution.
30 December 2004
puppy 1.4 out and about unslung 3.x in beta
Very few updates in the recent past, mainly due to birthdays, Christmas and flu.
Work on puppy has not stopped, in fact, puppy is now at version 1.4. I got sidetracked for a little while and collaborated on an EPG file format specification used to ingest 7 day EPG data to the Toppy. The result is a data converter that will convert JavaXMLTV object database to Topfield Guide Data (.tgd) file, which can then be uploaded to the Toppy using puppy and ingested by an EPG uploader TAP. This gives the user the ability to have a 7 day EPG uploaded to the Toppy.
It works well.
In the meantime, Unslung 3.x made it into beta. This means that the standard unslung firmware is now perfectly capable of supporting puppy out of the box. Users do not need to follow any special instructions for a puppy compatible NSLU2 install.
The other time consuming task has been answering questions from puppy users and potential users. Clearly the combination of NSLU2 and puppy is a great enabling technology with lots of potential. There are a lot of things that could be achieved, but I only have so much time on my hands.
The creation of puppy has been driven by my own needs and as soon as those needs are met, I tend to move on. I am quite comfortable with command line interfaces and shell scripts, so GUIs are very low on my list of priorities. (Not to mention that by the time I am done with my day job, I've had it up to here with designing and implementing GUIs :)
There's plenty of scope for other people to implement GUIs and other icing / usefull features on top of the work I have started. I would be very happy to co-operate with others to help them with whatever puppy related projects they may wish to undertake. After all, it will benefit everyone, including me. Interested developers should contact me - toppy at urbanec dot net
Work on puppy has not stopped, in fact, puppy is now at version 1.4. I got sidetracked for a little while and collaborated on an EPG file format specification used to ingest 7 day EPG data to the Toppy. The result is a data converter that will convert JavaXMLTV object database to Topfield Guide Data (.tgd) file, which can then be uploaded to the Toppy using puppy and ingested by an EPG uploader TAP. This gives the user the ability to have a 7 day EPG uploaded to the Toppy.
It works well.
In the meantime, Unslung 3.x made it into beta. This means that the standard unslung firmware is now perfectly capable of supporting puppy out of the box. Users do not need to follow any special instructions for a puppy compatible NSLU2 install.
The other time consuming task has been answering questions from puppy users and potential users. Clearly the combination of NSLU2 and puppy is a great enabling technology with lots of potential. There are a lot of things that could be achieved, but I only have so much time on my hands.
The creation of puppy has been driven by my own needs and as soon as those needs are met, I tend to move on. I am quite comfortable with command line interfaces and shell scripts, so GUIs are very low on my list of priorities. (Not to mention that by the time I am done with my day job, I've had it up to here with designing and implementing GUIs :)
There's plenty of scope for other people to implement GUIs and other icing / usefull features on top of the work I have started. I would be very happy to co-operate with others to help them with whatever puppy related projects they may wish to undertake. After all, it will benefit everyone, including me. Interested developers should contact me - toppy at urbanec dot net
16 December 2004
puppy 1.1 tagged.
The next beta installment has been tagged in CVS as
Once the change propagates to anonymous CVS, the unslung package will be updated and puppy 1.1 will be available for install to anyone with an NSLU2 running unslung firmware.
Changes from 1.0 to 1.1 include some internal cleanup of the code, implementation of delete and rename functions, device auto-detection, file permission setting for downloaded files, various packet tracing options, better error detection and reporting and code portability improvements.
Thank you to rwhitby and cazlar for their contributions / assistance.
PUPPY_1_1.
Once the change propagates to anonymous CVS, the unslung package will be updated and puppy 1.1 will be available for install to anyone with an NSLU2 running unslung firmware.
Changes from 1.0 to 1.1 include some internal cleanup of the code, implementation of delete and rename functions, device auto-detection, file permission setting for downloaded files, various packet tracing options, better error detection and reporting and code portability improvements.
Thank you to rwhitby and cazlar for their contributions / assistance.
15 December 2004
more features added.
Today, puppy gained a delete implementation that works and a rename implementation that does not. The rename bug is a rather nasty one and causes the Toppy to completely destroy the source file. The checked in change has rename disabled and it will print a warning.
I have my suspicions about the cause of this. I think it is related to the byte swapping algorithm and packet size adjustments required by it. I will address this issue as soon as I have the time. This could also be causing issues seen by x86 users trying to use puppy on Linux 2.6 desktops.
There have been a few other internal changes to some of the USB protocol handling. In particular, the protocol packet size constraints are now easier to follow and will actually cause an error, rather than issuing a command with truncated payload. This is unlikely to affect any legitimate use, since the check will only come into effect if the combined length of all command arguments in a packet reaches about 64kB.
Puppy is approaching a 1.1 release. Once the rename bug is sorted out, puppy will be tagged as
I have my suspicions about the cause of this. I think it is related to the byte swapping algorithm and packet size adjustments required by it. I will address this issue as soon as I have the time. This could also be causing issues seen by x86 users trying to use puppy on Linux 2.6 desktops.
There have been a few other internal changes to some of the USB protocol handling. In particular, the protocol packet size constraints are now easier to follow and will actually cause an error, rather than issuing a command with truncated payload. This is unlikely to affect any legitimate use, since the check will only come into effect if the combined length of all command arguments in a packet reaches about 64kB.
Puppy is approaching a 1.1 release. Once the rename bug is sorted out, puppy will be tagged as
PUPPY_1_1 and a new beta package will be pushed.
14 December 2004
troubleshooting enhancements.
puppy troubleshooting options have changed slightly.
I also created a new IRC channel so that other puppy developers can get near real-time help. Join #puppy on irc.freenode.net.
-v now shows verbose tracing output, but does not dump protocol packets.
-p will print packet headers - this should be sufficient to observe protocol state issues.
-P will dump the entire packet contents, both as hex and ASCII. This output could use some prettyfication. Patches gladly accepted!
I also created a new IRC channel so that other puppy developers can get near real-time help. Join #puppy on irc.freenode.net.
13 December 2004
puppy now in beta.
The alpha testing stage of puppy went well and puppy has gone from alpha to beta.
The puppy package has been added to the standard unslung package feed and can be installed simply by using
Please see the wiki for more information on what version of unslung firmware is required by puppy.
The puppy package has been added to the standard unslung package feed and can be installed simply by using
ipkg install puppy
Please see the wiki for more information on what version of unslung firmware is required by puppy.
11 December 2004
puppy unslung package in alpha.
It turns out that the issues with compiling puppy as an unslung package were related to compatibility issues between Linux 2.4 and Linux 2.6. I have backported puppy to work with Linux 2.4, while still maintaining compatibility with Linux 2.6.
The changes have been committed back to SourceForge CVS repository. The CVS tag
Alpha testers can build the puppy unslung package right now, by checking out the current version of unslung from CVS and then doing a
I'll be updating the puppy wiki page with more explicit instructions shortly.
The changes have been committed back to SourceForge CVS repository. The CVS tag
PUPPY_1_0 can be used to check out this initial version. The same tag will be used to build the puppy_1.0-1.ipk unslung package.
Alpha testers can build the puppy unslung package right now, by checking out the current version of unslung from CVS and then doing a
make puppy-ipk. As soon as I have a couple of reports of success, I will add puppy to the normal unslung package feed. Once that is done, puppy will be available for installation to anyone interested.
I'll be updating the puppy wiki page with more explicit instructions shortly.
09 December 2004
puppy unleashed.
The source code to puppy has been uploaded to the SourceForge CVS repository. Look for it at http://sf.net/projects/puppy
rwhitby started work on turning puppy into an unslung package, however there are some issues. I developed puppy using the OE cross-compiler and that works just fine. But when attempting to use the crosstool environment, as used for unslung packages, the compile process just blows up.
I'll take a look at the issues as soon as I can, but that may not be until the weekend. If anyone can help solve this issue, I'd be happy to accept patches. Please use the SourceForge facilities to send a patch.
rwhitby started work on turning puppy into an unslung package, however there are some issues. I developed puppy using the OE cross-compiler and that works just fine. But when attempting to use the crosstool environment, as used for unslung packages, the compile process just blows up.
I'll take a look at the issues as soon as I can, but that may not be until the weekend. If anyone can help solve this issue, I'd be happy to accept patches. Please use the SourceForge facilities to send a patch.
07 December 2004
Home for puppy found.
The name of the project is now set in stone. It is puppy. The new official home for the project is http://puppy.sf.net. The NSLU2-Linux project has kindly provided a whole new section on their wiki just for puppy. Special thanks must go to rwhitby for helping with all the administrative tasks.
The source code to puppy will be uploaded to the SourceForge CVS repository within the next week. It will be released under the GPL, but there are a few minor items to sort out first.
Puppy had quite a good work out over the last week and downloaded in excess of 100GB of data from my Toppy. All without any problems at all.
The source code to puppy will be uploaded to the SourceForge CVS repository within the next week. It will be released under the GPL, but there are a few minor items to sort out first.
Puppy had quite a good work out over the last week and downloaded in excess of 100GB of data from my Toppy. All without any problems at all.
30 November 2004
Answers to initial round of questions.
The announcement of this blog page has created some interest. As anticipated, the questions started coming in. Here are a few answers to some of the more pertinent questions.
Is this a Linux replacement for Altair?
Yes and no. The software I wrote is targeted at Linux running on an embedded ARM processor. The target ARM platform is running in big endian mode as opposed to little endian mode used by Linux on x86. There are other portability concerns to take into consideration, such as the way the compiler packs structures. I tried to write portable code, so making the software work under Linux on x86 should be trivial, but it is only tested on the NSLU2. I'm quite happy to leave the porting effort (if any at all) to others.
The software I wrote is not exactly a replacement for Altair. It does not have a GUI or all the capabilities of Altair. The long term plan is to add all the functionality that can be supported, such as turbo mode, file renaming, file deletion, etc.
I have no plans for a GUI, since the software needs to run on a headless embedded system. I expect that at some stage a web interface will be created to allow the user to initiate transfers between any drive on the network and the Toppy.
Should I buy the NSLU2 hardware now?
I like gadgets that serve a purpose. I like gadgets that can serve several purposes even better. The great thing about the Linksys NSLU2 is that it is a pretty handy device on it's own and it can be reprogrammed to do other things. If you can use a box that allows you to attach USB hard disks to your network, then by all means go ahead. Even if this project dies tonight, you will have a device that serves a purpose.
If you are a Linux hacker and want to get involved, then I would encourage you to get a NSLU2 now. You want to make sure that you have your toy in time for Christmas.
If your only purpose for the NSLU2 is to add an easy to use Ethernet interface to your Toppy, I would suggest that you wait until the project gains momentum. I expect that eventually it will be easy enough for a Windows user with minimal skills to get going, but we are nowhere near that goal just yet.
Where should I buy my NSLU2 from?
Since I don't have any incentive to recommend one vendor over another, I won't mention any one in particular. I am quite happy to recommend a particular retailer if we can structure a mutually beneficial relationship. If I do end up recommending someone in particular, I will justify the reason for the recommendation.
Where can I get a copy of this software?
You can't just yet. I need to take care of some house keeping before I unleash the project on the general public. House keeping includes such things as choosing a licence for the source (probably GPL), finding a home for the project (probably SourceForge) and of course writing developer documentation and guidelines. House keeping is tedious. I am not looking forward to it, but the plan is to get it done before Christmas.
Most importantly, I'll have to choose a name for the project. Up until now, the project has been code named puppy and unless a better name comes out of somewhere, it will stay. Suggestions are welcome, but please don't come up with sad aberrations of words like Linux, Tux, Slug or Altair.
When will it be finished?
Probably never. I expect the project to be adopted by the community and evolve indefinitely. There will always be more features that someone will want.
Can I be a beta tester?
Not yet. I am looking for others to join in the development effort. Part of the development cycle is setting up the environment, building the binaries from the source code and testing. At this stage, getting everything set up is non-trivial and a new comer will probably require a fair amount of help. Any effort I put into helping others with set up is effort taken away from making further progress on the project. That is why I am only interested in other developers at this stage. We need to get some initial momentum, so that there are enough people putting in the effort and energy to add features, improve the software and distribution model and eventually help end users with deployment.
If you want to get involved in development, start by getting yourself a NSLU2 and an external USB 2.0 hard disk. I currently use a 6GB laptop drive in a 2.5" drive enclosure powered by the NSLU2 and have a 36GB drive in a 3.5" externally powered enclosure just waiting to be used.
Is this a Linux replacement for Altair?
Yes and no. The software I wrote is targeted at Linux running on an embedded ARM processor. The target ARM platform is running in big endian mode as opposed to little endian mode used by Linux on x86. There are other portability concerns to take into consideration, such as the way the compiler packs structures. I tried to write portable code, so making the software work under Linux on x86 should be trivial, but it is only tested on the NSLU2. I'm quite happy to leave the porting effort (if any at all) to others.
The software I wrote is not exactly a replacement for Altair. It does not have a GUI or all the capabilities of Altair. The long term plan is to add all the functionality that can be supported, such as turbo mode, file renaming, file deletion, etc.
I have no plans for a GUI, since the software needs to run on a headless embedded system. I expect that at some stage a web interface will be created to allow the user to initiate transfers between any drive on the network and the Toppy.
Should I buy the NSLU2 hardware now?
I like gadgets that serve a purpose. I like gadgets that can serve several purposes even better. The great thing about the Linksys NSLU2 is that it is a pretty handy device on it's own and it can be reprogrammed to do other things. If you can use a box that allows you to attach USB hard disks to your network, then by all means go ahead. Even if this project dies tonight, you will have a device that serves a purpose.
If you are a Linux hacker and want to get involved, then I would encourage you to get a NSLU2 now. You want to make sure that you have your toy in time for Christmas.
If your only purpose for the NSLU2 is to add an easy to use Ethernet interface to your Toppy, I would suggest that you wait until the project gains momentum. I expect that eventually it will be easy enough for a Windows user with minimal skills to get going, but we are nowhere near that goal just yet.
Where should I buy my NSLU2 from?
Since I don't have any incentive to recommend one vendor over another, I won't mention any one in particular. I am quite happy to recommend a particular retailer if we can structure a mutually beneficial relationship. If I do end up recommending someone in particular, I will justify the reason for the recommendation.
Where can I get a copy of this software?
You can't just yet. I need to take care of some house keeping before I unleash the project on the general public. House keeping includes such things as choosing a licence for the source (probably GPL), finding a home for the project (probably SourceForge) and of course writing developer documentation and guidelines. House keeping is tedious. I am not looking forward to it, but the plan is to get it done before Christmas.
Most importantly, I'll have to choose a name for the project. Up until now, the project has been code named puppy and unless a better name comes out of somewhere, it will stay. Suggestions are welcome, but please don't come up with sad aberrations of words like Linux, Tux, Slug or Altair.
When will it be finished?
Probably never. I expect the project to be adopted by the community and evolve indefinitely. There will always be more features that someone will want.
Can I be a beta tester?
Not yet. I am looking for others to join in the development effort. Part of the development cycle is setting up the environment, building the binaries from the source code and testing. At this stage, getting everything set up is non-trivial and a new comer will probably require a fair amount of help. Any effort I put into helping others with set up is effort taken away from making further progress on the project. That is why I am only interested in other developers at this stage. We need to get some initial momentum, so that there are enough people putting in the effort and energy to add features, improve the software and distribution model and eventually help end users with deployment.
If you want to get involved in development, start by getting yourself a NSLU2 and an external USB 2.0 hard disk. I currently use a 6GB laptop drive in a 2.5" drive enclosure powered by the NSLU2 and have a 36GB drive in a 3.5" externally powered enclosure just waiting to be used.
29 November 2004
Adding an Ethernet interface to a Topfield TF5000PVRt
The Australian Topfield TF5000PVRt owner community has shown a great deal of interest in adding an Ethernet interface to their beloved PVR. While the USB 2.0 port on the Toppy is great, it requires a computer within about 5m of the device. Furthermore, because Topfield only provide a Windows application for communicating with the device, the PC must run either Windows 2000 or Windows XP - there are mixed reports of partial success with Windows 98. Given that the Toppy is most likely to be used in the loungeroom, with other home entertainment equipment, the physical proximity of a PC is not exactly practical or even desirable.
My solution to this problem is to use an embedded device that has an 10/100 Mbps Ethernet port to connect to the LAN and a host USB 2.0 port to connect to the Toppy. This takes care of the physical connections. There are several products available on the market that could do the job electrically, but just having the right plugs is not enough. The most important part in constructing a usable solution is the ability to add some smarts to this embedded device, so that it can talk the Toppy protocol and present the PVR on the network in a user friendly and platform independent way.
I've been on the look out for a suitable device for several months. When the Kuro Box was announced, I thought I may have finally found a suitable solution. Unfortunately, the device was fairly expensive and only available in Japan and North America. Importing the Kuro Box to Australia really wasn't an option, because the built in power supply would not work in a 240V country.
As luck would have it, while researching the Kuro Box, I stumbled on information about the Linksys NSLU2. This device looked exactly like the thing I was looking for. It is available locally in Australia for about AUD$170, has two USB 2.0 ports, one 10/100 Mbps Ethernet port and runs embedded Linux. What is even better, is the fact that there is a very active developer community doing some excellent work on turning the NSLU2 into a very flexible, general purpose, embedded Linux platform.
I got involved with the NSLU2 development community and started contributing to the development even before I purchased my NSLU2. The NSLU2 is affectionately known in the development circles as the slug. While waiting for the delivery of my NSLU2, I started learning about the development environment used for Unslung and OpenSlug. I contributed the libusb packages before I got my slug. Unfortunately, the libusb library is broken in a number of ways and the released packages only appear to work superficially. The excursion into libusb source has cost me a couple of weeks in wasted effort, first trying to identify the problems, then attempting to rectify them. In the end, I decided to throw libusb out of the equation and wrote my application to work with the Linux kernel usbdevfs API directly.
While attempting to bring generic USB capabilities to Unslung 2.x, I also realised that the kernel did not have support for usbdevfs enabled. On top of that, I found an incompatibility between the NSLU2 standard kernel and the X's Drive II. The changes required to fix these problems, as well as other factors discussed in the NSLU2 community, led to the birth of a new Unslung variant named Unslung-able. With the help of rwhitby, I made some changes to the Unslung OE build process to enable a mechanism for creating distribution variants. Shortly after that, the NSLU2 development community decided to concentrate on only two variants, Unslung-standard and Unslung-able.
Once I had my NSLU2 platform configured to enable the USB communication with the Toppy, I started the development of a Toppy specific application to transfer data over USB. This is the component that replaces the Windows specific Altair application. I already had the Topfield documentation and as luck would have it, there was a Home Entertainment Expo at Darling Harbour. Topfield Australia were a major exhibitor, showing their existing product range as well as the rumoured Topfield TF7000PVRt. I turned up during the initial trade and press session, before the show was open to general public. I had a brief conversation with a couple of Topfield employees and a very long and productive chat with the general manager of Topfield Australia. I outlined to him a couple of the projects I had in mind for the TF5000PVRt and he was very supportive. He went as far as to offer to arrange for me to meet with a member of the Korean team that was going to be in Sydney later that day. Unfortunately, prior engagements prevented me from grabbing this opportunity. The good news was that I had a promise of co-operation and a personal invitation to contact the GM of Topfield Australia if I needed help with my projects. This unusually friendly and co-operative response fueled my ambition to succeed.
Over the next couple of weeks I spent most of my free time writing and debugging code for communicating with the Toppy. At one stage I ran into a brick wall with the CRC16 algorithm used by the Topfield USB protocol. I sent an email to the general manager of Topfield Australia at around 1am, requesting help. By about 3pm on the same day, I had the answer and the source code to the relevant implementation, presumably obtained directly from the Topfield engineers in Korea.
As of today, I have a proof of concept application that can:
I tested both uploads and downloads while using the Toppy for normal viewing. It works. I noticed dropped video frames and even rare audio drop outs during the playback of a previously recorded program, while at the same time recording two channels and also downloading a file from the Toppy. This is not really surprising because the Toppy is utilising an aggregate I/O bandwidth in excess of 80Mbps to cater for the four concurrent streams. I suspect that the biggest issue is in fact the performance of the polled USB port on the Toppy. The USB 2.0 implementation on the TF5000PVRt seems to be the weakest point of the unit.
The next steps are to get enough information together to allow others to follow in my footsteps and build on top of what I have done. I eventually expect this project to get to a state where a user can purchase an NSLU2, download a firmware image, flash the NSLU2 with this firmware image using a web browser UI and end up with a new device intended as a home network gateway for the Toppy. Eventually, I expect that this Toppy gateway will provide the user with a web interface that allows advanced operations, such as data migration of recorded files to other computers on the network, EPG data uploads and even remote user interface, to allow for such things as starting a recording from your WAP phone. While I don't expect to be building all of this by myself, I believe that putting in the ground work to provide the basic connectivity will allow others to do this.
To be continued...
My solution to this problem is to use an embedded device that has an 10/100 Mbps Ethernet port to connect to the LAN and a host USB 2.0 port to connect to the Toppy. This takes care of the physical connections. There are several products available on the market that could do the job electrically, but just having the right plugs is not enough. The most important part in constructing a usable solution is the ability to add some smarts to this embedded device, so that it can talk the Toppy protocol and present the PVR on the network in a user friendly and platform independent way.
I've been on the look out for a suitable device for several months. When the Kuro Box was announced, I thought I may have finally found a suitable solution. Unfortunately, the device was fairly expensive and only available in Japan and North America. Importing the Kuro Box to Australia really wasn't an option, because the built in power supply would not work in a 240V country.
As luck would have it, while researching the Kuro Box, I stumbled on information about the Linksys NSLU2. This device looked exactly like the thing I was looking for. It is available locally in Australia for about AUD$170, has two USB 2.0 ports, one 10/100 Mbps Ethernet port and runs embedded Linux. What is even better, is the fact that there is a very active developer community doing some excellent work on turning the NSLU2 into a very flexible, general purpose, embedded Linux platform.
I got involved with the NSLU2 development community and started contributing to the development even before I purchased my NSLU2. The NSLU2 is affectionately known in the development circles as the slug. While waiting for the delivery of my NSLU2, I started learning about the development environment used for Unslung and OpenSlug. I contributed the libusb packages before I got my slug. Unfortunately, the libusb library is broken in a number of ways and the released packages only appear to work superficially. The excursion into libusb source has cost me a couple of weeks in wasted effort, first trying to identify the problems, then attempting to rectify them. In the end, I decided to throw libusb out of the equation and wrote my application to work with the Linux kernel usbdevfs API directly.
While attempting to bring generic USB capabilities to Unslung 2.x, I also realised that the kernel did not have support for usbdevfs enabled. On top of that, I found an incompatibility between the NSLU2 standard kernel and the X's Drive II. The changes required to fix these problems, as well as other factors discussed in the NSLU2 community, led to the birth of a new Unslung variant named Unslung-able. With the help of rwhitby, I made some changes to the Unslung OE build process to enable a mechanism for creating distribution variants. Shortly after that, the NSLU2 development community decided to concentrate on only two variants, Unslung-standard and Unslung-able.
Once I had my NSLU2 platform configured to enable the USB communication with the Toppy, I started the development of a Toppy specific application to transfer data over USB. This is the component that replaces the Windows specific Altair application. I already had the Topfield documentation and as luck would have it, there was a Home Entertainment Expo at Darling Harbour. Topfield Australia were a major exhibitor, showing their existing product range as well as the rumoured Topfield TF7000PVRt. I turned up during the initial trade and press session, before the show was open to general public. I had a brief conversation with a couple of Topfield employees and a very long and productive chat with the general manager of Topfield Australia. I outlined to him a couple of the projects I had in mind for the TF5000PVRt and he was very supportive. He went as far as to offer to arrange for me to meet with a member of the Korean team that was going to be in Sydney later that day. Unfortunately, prior engagements prevented me from grabbing this opportunity. The good news was that I had a promise of co-operation and a personal invitation to contact the GM of Topfield Australia if I needed help with my projects. This unusually friendly and co-operative response fueled my ambition to succeed.
Over the next couple of weeks I spent most of my free time writing and debugging code for communicating with the Toppy. At one stage I ran into a brick wall with the CRC16 algorithm used by the Topfield USB protocol. I sent an email to the general manager of Topfield Australia at around 1am, requesting help. By about 3pm on the same day, I had the answer and the source code to the relevant implementation, presumably obtained directly from the Topfield engineers in Korea.
As of today, I have a proof of concept application that can:
- show the size of the Toppy hard disk and the amount of available space.
- list the contents of directories on the Toppy.
- upload files of any size to the Toppy.
- download files of any size from the Toppy.
- abort an unsuccessful upload or download.
- reboot the Toppy (untested).
I tested both uploads and downloads while using the Toppy for normal viewing. It works. I noticed dropped video frames and even rare audio drop outs during the playback of a previously recorded program, while at the same time recording two channels and also downloading a file from the Toppy. This is not really surprising because the Toppy is utilising an aggregate I/O bandwidth in excess of 80Mbps to cater for the four concurrent streams. I suspect that the biggest issue is in fact the performance of the polled USB port on the Toppy. The USB 2.0 implementation on the TF5000PVRt seems to be the weakest point of the unit.
The next steps are to get enough information together to allow others to follow in my footsteps and build on top of what I have done. I eventually expect this project to get to a state where a user can purchase an NSLU2, download a firmware image, flash the NSLU2 with this firmware image using a web browser UI and end up with a new device intended as a home network gateway for the Toppy. Eventually, I expect that this Toppy gateway will provide the user with a web interface that allows advanced operations, such as data migration of recorded files to other computers on the network, EPG data uploads and even remote user interface, to allow for such things as starting a recording from your WAP phone. While I don't expect to be building all of this by myself, I believe that putting in the ground work to provide the basic connectivity will allow others to do this.
To be continued...
Subscribe to:
Posts (Atom)