Mark’s Braindump

hmac-sha1

Filed under: Code — mark @ 2:48 am

I wrote a hmac-sha1 function based on FIPS-198a which I needed, and I’ve randomly decided to MIT license it in case anybody else needs one. The code is located here; there’s also a zip file with all the source code and a visual studio solution in it. Tested on debian etch amd64 (g++ -o hmac hmac.cpp sha1.cpp) and windows. Note that the sha1 implementation included is pinched from somewhere else and is 3-clause BSD — it would be easy enough to replace with another however. Enjoy!

Comments Off

Goodbye, Microsoft…

Filed under: Random — mark @ 7:37 pm

(…well, sort of, I still have to write code that runs on Windows as a day job — I’m looking to correct that in the medium term though!)

I finally got around to installing linux on my laptop. I’m a debian person generally (for various reasons, but I don’t want to provide any more distribution war fodder), but debian are a bit slow to change. I want to mess around with shiny new applications, so I plumped for ubuntu feisty. I’m right at home with it having used debian on servers for years. And I love it. Been going for about a month, and haven’t looked back (apart from the vmware image of Windows needed for work). I am amazed how far linux has come on the desktop. I’ve had not particularly technical people use my laptop, and _not even notice_ that it wasn’t running an OS that they know. Next task is to get stuck into fixing bugs, I’ve run into a few but then that’s why I picked an unstable release
Oh, and if you want to know, the combination of circumstances that got me to get off my arse and help with ubuntu bug #1 were:

- New hard disk for the laptop (=> spare to put OS on to play with
- The arrival of Windows Vista with all of its great new features

Hurrah for me, I did something good for a change!

Comments (0)

ARC-1160 SATA RAID updated

Filed under: Random — mark @ 7:25 pm

Figured I might as well use this piece of software since I have to maintain it.

Basically to summarise my experiences with SATA RAID controllers (see here for the full details) under a basic stress load:

1. Adaptec 21610SA. Junk, very slow, locks up after about 4 hours.
2. 3ware 9550. Kicks drives out after about 10 minutes.

Then I got an Areca 1160. This works, kind of. It corrupts data, this is directly correlated with a ‘media error count’ that is visible only on the individual, detailed status page for each drive available through the card’s HTTP interface. After getting in touch with Areca they advised me that this was either power (I tried it with a pair of PSUs with the load split roughly evenly, so I don’t think that this is really the case), or it’s an issue with the firmware of the drives (16xWD320RE SATA disks).

However, entirely turning off the write cache makes the thing work OK, and it has done this for about a year. The write speed is absolutely pitiful though (and writes also kill reads). I get ~15Mb/s in moderately seek bound workloads, and ~30Mb/s for sequential writes. Hurrah, slower than some of my really old single IDE disks! Oh well, it’s good enough for a backup/TV server at any rate. Overall SATA RAID is still totally pants though!

Comments (0)

SATA RAID is rubbish

Filed under: Random — mark @ 1:00 am

For the last 4 months, I’ve had a 4TB storage array (=16×320Gb WD RE SATA hard disks) sitting in my front room.

Doing nothing. Nothing at all.

The reason? SATA RAID cards are _unfeasibly_ bad. Here’s what I’ve tried so far, under linux (various kernel versions and distributions):

- an Adaptec 21610SA. This lasts about 4 hours before it does the classic aacraid: scsi hang? (google for it, lots of references). Adaptec sent me a “heat sync” (sic.) for some reason, in a russian doll style series of packaging.
- a 3ware 9550. This lasts about 10 minutes before it powers down a random drive (physically: you can hear it), then claims that the drive has been reset (well, duh). 3ware deny everything (the last thing they told me was that it was “noise on the PCI bus”. The 3ware card has been tried in a couple of different motherboards, same thing in both of them).

Is this isolated? From friends I know of:

- At least two other machines containing Adaptec AAC series cards with the scsi hang? issue;
- At least two other machines containing 3ware 9xxx cards that pop drives;
- At least one other machine containing a 3ware 9xxx card that pops drives under a different OS

Good news? I got to play with an Areca-1160. Managed 24 hours without an issue, so that’s the best so far. If I can find someone who distributes them in the UK, I’ll get one!

Random technical details:

- I break the cards with dd if=/dev/sdX of=/dev/null (x10); dd if=/dev/zero of=fooX (x10); bonnie++ (x10); and cp -a on a kernel tree (x10). IoW, just a bit of IO.
- The cards were tried in a P8SCi motherboard, 3Ghz xeon, plenty of power supply (including 1100W worth for some experiments)
- The P8SCi is on 3ware’s compatibility list
- 3ware card also tried in an Athlon64+H8SSL motherboard setup with a 760W power supply

I can’t believe these companies get away with peddling such crap. But then I suppose so do Highpoint and Promise to name a few I’ve seen shite (ATA-100/ATA-66) hardware off in the past.

Is SCSI any better?!

Comments (8)

ClamAV Windows Port updated to 0.88.1

Filed under: Random — mark @ 12:47 am

Updated to latest version (0.88.1). I’ve updated in the interm; will try to post other updates in a timely fashion. Do let me know if you are using this, that will be an incentive to do so.

Other fixes are:

- Updated to build with VC8 (VS2005)
- Fix a number of extraneous close() calls that VC8 doesn’t like
- Fix service stub so that it works with w2k3

You can get the MSI installer & patch here.

Full source code can be downloaded here.

Comments (1)

LIRC PVR-150 IR blaster support, version 3

Filed under: Random — mark @ 10:11 pm

News

Latest: added a version that compiles with 2.6.27 kernels.
Latest: if you are packaging this, see the note at the bottom as to why I don’t provide a simple patch.
Latest: added an updated version that compiles with 2.6.24 kernels, tested against 2.6.23.12
Latest: updated `firmware` to a later revision from hauppauge (743 codesets)

HOWTO

This is not for the MCE version of the PVR-150. This includes a USB unit which does not work the same way as the normal PVR-150, use standard lirc (and lirc_mceusb) for that.
Use a recent kernel (note: the ivtv drivers have been part of the mainline kernel for a good number of revisions so no external drivers are required — sorry, I don’t know off hand which revision they were merged in), or for older kernels install ivtv-0.4.2+ (from this page). Earlier versions of ivtv _are not supported_.
Get the pre-patched lirc 0.8.5-CVS-pvr150 tarball. There are also earlier versions that can be found here, should you want them. The previous revision may be required for kernels < 2.6.27 (as it is untested on lower revisions -- in theory it should work)
You need the dialog package installed to use the lirc configuration GUI, so install that (apt-get install dialog, yum install dialog, whatever is appropriate for your distribution).
Unpack the patched lirc:
cd /usr/src tar xfj lirc-0.8.3-CVS-pvr150-2.tar.bz2 cd lirc-0.8.3-CVS-pvr150-2 ./setup.sh

Choose:
TV card
i - Hauppauge PVR-150 TV card (note: _NOT_ ‘g - Hauppauge TV card’)
Save configuration & run configure

then:

make && make install.
IR blaster only: Now you need the ‘firmware’. This is a set of data blocks that correspond to those generated by the windows software. This goes in /usr/lib/hotplug/firmware on my debian system. Depending on your system this might also be /usr/local/lib/firmware, /lib/firmware or /lib/modules.
Note that the entire firmware is kept in memory (currently 300K) so this makes the driver quite large. (I have no plans to sort this out, memory is cheap).
Check everything is working so far:
modprobe lirc_dev debug=1 && modprobe lirc_pvr150 debug=1

Check the syslog output. This should report something like:

Aug 28 02:09:11 soapbox kernel: lirc_pvr150: chip found with RX and TX
Aug 28 02:09:11 soapbox kernel: ivtv: i2c attach [client=Hauppauge PVR150 IR RX, ok]
Aug 28 02:09:11 soapbox kernel: ivtv: i2c attach [client=Hauppauge PVR150 IR TX, ok]
Aug 28 02:09:11 soapbox kernel: lirc_dev: lirc_register_plugin: sample_rate: 0
Aug 28 02:09:11 soapbox udev[5221]: creating device node ‘/dev/lirc0′
Aug 28 02:09:11 soapbox kernel: lirc_pvr150: firmware of size 302355 loaded
Aug 28 02:09:11 soapbox kernel: lirc_pvr150: 743 codesets loaded
Aug 28 02:09:11 soapbox kernel: lirc_pvr150: Hauppauge PVR-150 IR blaster: firmware version 1.3.0

This means that the driver has detected and initialised the IR blaster hardware — if you don’t see that then let me know.
You need to configure lircd, and find out which codeset you are going to be using. The easiest way is to start with this configuration file which contains key definitions for everything in the database. Do not use other lirc configuration files for specific STBs — these simply will not work. The IR chip is only capable of sending those codes which are in the database.
Start lircd. Note: if you are using a static /dev, you may need to make a device for lirc. If unsure, once you have verified that the module has been loaded ok, run ls -l /dev/lirc*. If you don’t see a /dev/lirc0 or similar, then try mknod /dev/lirc0 c 61 0 if the steps below fail.
modprobe lirc_dev && modprobe lirc_pvr150 debug=1 lircd --device=/dev/lirc0
You can now check if the remote is working using irw. Run this, and press buttons on the remote. You should see some output like
0000000000001795 00 Down Hauppauge_350.
Next, for the ir blaster you need to work out which codeset to use, this is the tricky bit. For this I have send_power_new, a script that just sends the power command in every single codeset. You may find your codeset number listed here if you are lucky.
Firstly, check that you are seeing the IR blaster blink. If you don’t have blinking lights at this stage, your cable probably isn’t in the card properly (try wiggling it around), or it may be broken.

Next you need to stick the IR blaster on the IR receiver of box that you intend to control, being quite careful to position it correctly — it has a very short range (a few cm) and took me a couple of goes to get right. The best way to do this is to find the IR demodulator on the box — easiest with a torch. Note that this is _not_ the light that comes on when you press a button on the remote, they tend to look like this.

If you can’t get this to work, please try and check it against the Windows driver if possible. If your device will work with the windows driver but not my driver, then it’s a driver bug that I should be able to fix. If it does not work with the Windows driver either then your only options are to use a different IR blaster or bug hauppauge until they add support for the box you are trying to control. At that point, I can update my
database too.
Once you know which codeset you want you can go and delete all of the rest from lircd.conf. They are named “XXX_key” so should be pretty easy to find. I also gave the keys standard names (0-9).
To get mythtv to work, configure a channel change script for your device. There’s one here that should work out of the box if you
rename the number keys.
If you’re happy, you can always send me beer money. If not, add comments at the bottom

That’s it, good luck!

Packaging

The lirc distribution tarball is generated using `make dist-bzip2` which uses the gnu autotools to generate a configure script, and Makefile.in, etc. The contents of these generated files depends very much on the version of autotools that is installed; this varies from distribution to distribution. I don’t have the same auto* as the lirc maintainers, so producing a patch file against a distribution tarball makes a patch bigger than the original source archive. Hence I don’t bother; I just make a new dist bzip2 and drop it here.

I maintain the code by importing a current lirc CVS into a subversion repository hosted on this server. I tag each lirc import, generate a diff from the previous lirc import and apply it to my source tree, then port any fixes from lirc_i2c.c to lirc_pvr150.c, test, and generate a new .tar.bz2. To get the source tree you can do:

svn co http://svn.blushingpenguin.com/svn/trunk/3rdparty/lirc lirc

and you can get the changes I made to the CVS revision of lirc with:

svn diff http://svn.blushingpenguin.com/svn/vendor/lirc/current http://svn.blushingpenguin.com/svn/trunk/3rdparty/lirc

Comments (364)

PVR-150 IR stops responding (fixed!)

Filed under: Random — mark @ 7:42 am

Update
This patch, with some fixes, has gone into the 0.4.2 release of ivtv. Please use ivtv 0.4.2+ rather than following the instructions here. You can get this from the ivtv site.

History
This is related to my IR blaster driver, in case that’s what you are looking for. Do read on though, it might be informative! If you are just looking for patch instructions, dive right to the end to skip all this detail.

A while ago I looked around to see if the hauppauge windows driver for the remote control had any issues. I found quite a few people who claimed that it stopped responding, and that they had to reset the hauppauge software, etc and one or two pointers to a new (beta) driver on the hauppauge site. This was fairly intriguing as among the release notes it lists “i2c (IR blaster) fixes”. I managed to find time this weekend to take the driver apart, and see exactly what they did. Basically they rewrote the i2c support.

Some background, in case you aren’t familar: i2c is a low speed (varies, say ~100-400KHz) 2 wire serial bus for interconnecting various random chips. Because it’s so useful, it actually gets used all over the place to ‘glue’ various chips together on a board and then let the OS driver control them. For example, on your typical PVR-150 card you’ll find at least tuner, video encoder, IR chip and eeprom (configuration data). These are all i2c connected, and the driver talks i2c to them to program them (as well as other memory mapped registers, but let’s stick to the issue).

In some cases, the i2c protocol is implemented by special hardware on one of the chips (e.g. the bt878a chip has an i2c controller in it). This is the nice case, since the CPU doesn’t have to do much in the way of work. In the cheap case, you get to control the clock & data lines yourself from the OS (a kind of winmodem approach!). The hauppauge stuff takes the latter approach — indeed so does a bunch of other hardware. It’s so common that there is a module in the linux i2c subsystem specifically for doing this — it’s called i2c-algo-bit, and the approach is known as ‘bit-banging’. As a side issue, bit-banging takes up some constant fraction of CPU, because it has to use busy delays (it’s not possible to get microsecond accurate sleeps in the kernel, so we must run the CPU in a loop). Since these loops are constant _time_ you’ll find that whatever the CPU the IR polling (for example) takes a constant amount of time. For lirc_i2c, this works out at about 1%, and for my lirc_pvr150 module (that sends a ‘poll’ command to the receiver as well to see if it is borked or not), it’s about 2%. This is why I think the hardware approach is better, but whatever, we are stuck with it.

From the looks of the hauppauge driver changes it appears that the classical bit banging algorithm causes problems with the IR chip that the PVR-150 uses. I disassembled both the new and old drivers, and the older driver uses something very similar to the approach of i2c-algo-bit — it uses KeStallExecutionProcessor, or a busy loop, to implement delays. The newer driver dispenses with this, and instead polls the lines to see when they have become high/low and uses memory mapped register reads for stalling. I’m guessing that those reads are (roughly) constant time because they have to go across the PCI bus. I implemented the newer method, and low and behold this seems to solve my problems. I’ve not seen an IR chip reset by my lirc_pvr150 driver in 16 hours now (update: now 28), whereas I was getting them _much_ more frequently beforehand (dozens a day). This stops the remote dropping out if the chip has to reset, which it did do a few times a day for seconds at a time. So I’m happier. Anyway, if anyone else would like to try it, here’s the HOWTO part…

The HOWTO part

This patch is against the latest SVN version of ivtv. Things are changing rapidly with ivtv, and there is currently no rea

Mark’s Braindump

December 2, 2008