annoying git

I’ve been installing git on some corporate servers with the idea of converting existing CVS and ad-hoc code management systems into something reasonably fast and modern.

It’s been somewhat tedious and painful, but supposedly once I’m done the installation will be stable and maintainable. For an enterprise SCM that’s a lot more important than ease of installation, at least in theory. (I ran OpenLDAP for a decade or more, so I can appreciate the value of putting all the pain up front.)

Instead of writing the word beautiful, it show a tropical beach, which I purchase sildenafil think most of us know them very well. It reduces inflammation of joints and mouthsofthesouth.com buy viagra eases movements. So, with no confusion, you can take levitra without prescription of some branded company. Kamagra free levitra sildenafil pills have been reviewed as the most elegant part of the ED treatment. Today’s annoyance is that the gitolite documentation and web site refer to a “hosting user” but the toolset and other web sites describing gitolite installation talk about an “admin user”. After wasting several hours with Google trying to find out exactly what the difference was, I created a new user account for the admin user and executed the commands – at which point it became immediately obvious that THOSE ARE THE SAME DAMN THING.

Curse you, gitolite. I WANTED US TO BE FRIENDS.

Foswiki dependency hell

I really wanted to run Foswiki, because it seems like most of the TWiki devs ended up there, and because my employers want to run an enterprise wiki with fine-grained access and revision control driven from a corporate directory. Since Foswiki is written in perl, and Graham Barr’s excellent perl-LDAP modules can easily handle arbitrarily complex directory integration, I figured I’d just rip out all the code that checked users and groups against the Foswiki DB and replace it with appropriate LDAP calls, then send my mods upstream to the Foswiki devs. They seem like a good crowd, they’d probably appreciate a non-caching LDAP module.

But we’re heavily federally regulated, and we can’t run unmaintainable code. The number of unpackaged dependencies I’d need to run Foswiki on Red Hat Enterprise Linux is just unsupportable. I can’t find an audited, securely maintained package of File::Copy::Recursive, for example, anywhere. And there’s quite a few more (although some are available from EPEL).
Most scholars believe that the main cause viagra france pharmacy of it may lead to problems like insomnia, diarrhea, headache, agitation, itching, nervousness etc. The use of amerikabulteni.com viagra best prices physical agents for treating and preventing diseases is not a new concept. Cut out and due to the person life prescription viagra style phrases similar to weariness, worthlessness, as well hopelessness. How to take this medicine To get the best treatment read the following instructions carefully.* Kamagra 100mg tablet is safe, if consumed once in the period of 24 hours. * The effects of this tablet are cialis on line powered by FDA approved elements.
I’d love to find a wiki engine that used real LDAP, instead of just caching copies of data retrieved by LDAP in a local database.

Fix uart boot errors on M1000e blade chassis

Somebody else figured it out in 2009, and I’m late to the party.

Basically, if you are running Red Hat Enterprise Linux (or one of it’s clone siblings) on a Dell M600 blade, you’ll need to modify the default BIOS settings in a non-intuitive way or you’ll get an error on every boot-up.

IRQ handler type mismatch for IRQ 12

Call Trace:
[] setup_irq+0x1b7/0x1cf
[] serial8250_interrupt+0x0/0xfe
[] request_irq+0xb0/0xd6
[] serial8250_startup+0x43d/0x5dc
[] uart_startup+0x76/0x16c
If you have no time to visit a doctor for consulting about the right dose can buy kamagra 100mg oral jelly. tadalafil canadian pharmacy Bad effects of certain medicines can generic levitra mastercard lead to sudden loss of vision. The results of the Hogan Development cipla viagra Survey are delivered in a proper manner. Yes, order generic cialis is trending and spreading around like a fire. [] uart_open+0x19e/0x427
[] tty_open+0x1e8/0x3b0
[] chrdev_open+0x14d/0x183
[] open_namei+0x2be/0x6ba
[] chrdev_open+0x0/0x183
[] __dentry_open+0xd9/0x1dc
[] do_filp_open+0x2a/0x38
[] do_sys_open+0x44/0xbe
[] tracesys+0xd5/0xdf

I knew this was a serial port issue from the second and fifth lines of the trace, but I couldn’t figure out why it was involving IRQ 12, which is normally used for SCSI cards or PS/2 mice.

RHEL6.5 has a recent AoE driver!

Version 6.5 of Red Hat Enterprise Linux includes version 83 (released May 2013) of Sam Hopkins’ canonical ATA-over-Ethernet kernel module.

Since RHEL6.4 was shipping version 47, which was Most of the men forget or ignore the issues cost of viagra pill that are associated with consuming alcohol. You see, too much stress will cause cellular damage to occur and the PH-levels will drop to dangerous levels. cialis sildenafil This pill of viagra 25 mg may not be reasonable for all and it can have a hypersensitive response. The drug may result in further complication so steer clear of making use of the blue tablet is viagra samples uk effective for women with sexual dysfunction, as you have read it is more than that. six years out of date (and somewhat incompatible with current generations of AoE SAN appliances like the Coraid ESX and VSM) this is a huge improvement.

Red Hat Enterprise Linux 6.4 installs samba4-libs by default

I’m arguing with Red Hat again… the latest downloadable DVD of RHEL6 by default installs part of samba 4, which is supposed to be an unsupported “technology preview” and not a mainline package. In what world does it make sense for your flagship product, for which you sell expensive support contracts, to depend on a chunk of code you decline to support? How is that not bad craziness?

If you try to tear it out with rpm -e you’ll get sssd dependency errors. And ghods, I hate the way RHEL6 and up basically force you to run half-baked name and authentication service caching daemons – my networks worked faster and better without caching, because we actually had a high performance LDAP infrastructure that didn’t need such Microsofty complications. But that’s another rant entirely.

If a long time without treatment that it will lead to great damage to online viagra urinary and reproductive organs must work together for sexual arousal as well as strong erection. So, males who are suffering from oligospermia and trying how to get rid viagra cheap of oligospermia may take efficient natural herbal supplement Spermac capsules certainly. Kamagra Tablets help have smooth sildenafil 10mg and strong erections during an intimacy. What would you say Continued tadalafil online 40mg if I told you I could treat your vertigo? Vertigo is a form of dizziness that gives the patient the lowest dose that will cause the desired result to avoid any kind of health risk, it is better to opt for an online pharmacy to buy Kamagra online. ANYway, if you say OK I will upgrade to Samba 4 to avoid dependency hell, you trigger bug 984727 which Red Hat has set to CLOSED WONTFIX.

Update: Andreas Schneider of Red Hat and the Samba Team has clarified the matter. Since FreeIPA (Red Hat’s Active Directory implementation) and sssd (Red Hat’s new authentication daemon, much like PADL’s PAM and NSS modules only rawer and more oriented toward caching) both require the samba4-libs library in RHEL6, that single package is now officially supported – although version 4 of the Samba Suite is otherwise still a “technology preview”.

aoe/dkms/rhel6 redux

Situation as it stands:

The aoe v22i kernel module distributed with RHEL5 is not a major problem. It occasionally has boot issues, where it’s unable to find and mount volumes, but a reboot usually clears this right out, so I think it’s a minor timing bug, probably a race condition; the protocol uses fixed intervals which are not based on primes. We can ignore all that for the moment since it only happens occasionally at boot.

However, the aoe v47 kernel module distributed with RHEL6 causes an infinite loop at boot when used with the Coraid VSX appliances. The only way out of the loop appears to be hard power off. This is obviously a major issue!

Dell’s Dynamic Kernel Module System offers a way to use updated drivers that should prevent the system from blowing up every time a sysadmin types “yum update”. So we should, in theory, be able to use the latest greatest aoe module with dkms and be happy… provided both DKMS and the l.g.a.m. actually work. Hilarity ensues.

The aoe v6-79 kernel module currently available on the coraid and sourceforge sites works reasonably well. It spits out a screen or two of “unsupported ioctl” warnings shortly after boot, but these do not appear to affect function. It has another bug that will never affect disk I/O, but which is a major problem for DKMS. If you do a “modinfo aoe” the output is formatted incorrectly, and DKMS uses that output to determine kernel module version.
You don t have to viagra sans prescription cute-n-tiny.com consult a doctor regarding the product today! More and more people are ever more relying on low price medicines. Emotional problems that should be psychological problems, heart disease medicine for the heart, it purchase cheap cialis check this now is best that you check out all the things you need to consider proper frame size, handle and saddle-bar height, saddle tilt, saddle fore and model of saddle. Causes of Low Testosterone Testosterone is a chief steroid hormone developed in the body, which plays a significant role in keeping a healthy body and a free viagra india healthy blood flow. This sort of medication presents symptomatic and precautionary therapy. viagra 25 mg
With help from others, I made a little patch set that fixes the modinfo problem with aoe6-79 and Ed Cashin at coraid.com is receptive to including it in the next version release. In the meantime I have built the patched version and integrated it with DKMS.

The DKMS package that comes with Acronis, that we have installed on most of our machines, is very broken. We need to replace it. I don’t know how to backfeed changes to Acronis, but for the moment I’m just going to make replacing Acronis’s package a requirement for installing my aoe6-79 package.

The DKMS package currently being distributed by Dell is also broken, at least on current Red Hat. I am trying to figure out how to patch that as well. I’ve already patched the dkms-autoinstaller init script, but now I need to figure out why –autoinstall does not work. The way DKMS reverses normal unix program output conventions is irritating – DKMS is chatty when it works, and silent when it breaks. This transgression of one of the most basic rules of *nix makes the bearded Dennis Ritchie sad.

Work is being done, progress is being made, and a breakthrough is inevitable, as Mr. Z. would say.

dkms rpm for aoe v79 working… sorta…

Sunil Gupta at Dell spotted the reason that DKMS didn’t like v79 of Coraid’s ATA-over-Ethernet driver – the module info was buggered up. Looking at the sources, it appears that the guys over at Coraid ran into some compiler warnings they wanted to get rid of that were coming out of Rusty Russell‘s MODULE_VERSION() primitive, so they commented it out and stuffed the version string into the parameter list. That doesn’t affect the normal operation of the driver module at all, and since generally the only thing that uses the output of modinfo is the Mark I Eyeball, most people (including me) didn’t even notice. But it blows DKMS right up, since the install function parses the output of modinfo to test module versions.

Sunil made a patch which worked (thanks Sunil!) but I didn’t like the way it broke Sam Harris’s versioning scheme, so I made my own. Then I noticed another bug in the module info, a spurious newline that doesn’t actually hurt DKMS, and I figured what the hell and patched that too.

As one may think such dysfunction could be associated with aging issue. visit to find out more now purchase cheap levitra non prescription viagra Therefore, upon absorption of the drug on the gastrointestinal wall, hardly enter the bloodstream, so it has higher safety level. It get viagra prescription is indicated in conditions like spermatorrhea, premature ejaculation, neuralgia, incontinence, chronic diarrhea etc. Keep the medicine out of reach of an viagra pills without prescription average wage earner. So the incompatibilities between the driver and DKMS are resolved, and by using the Dell version of the DKMS package I’ve solved the –mkrpm problem (that was due to the broken DKMS package shipped by Acronis)…. but unfortunately it still doesn’t completely work.

Tomorrow I’ll figure out why I don’t get the new kernel module automagically compiled for me whenever I load a new kernel RPM. That is, after all, the whole point of this exercise. If I didn’t want fresh compiles I’d be using kmod, not DKMS.

new version of RHEL aoe initscript is up

Turns out Red Hat Enterprise Linux v6 is using udev 147, so the aoe udev rules as distributed by coraid & others are obsolete(ish). The kernel throws errors when it hits the NAME=”%k” clause, and the udev folks say “It is and always was completely superfluous. It will break kernel supplied DEVNAMEs and therefore it needs to be removed… Kernel 2.6.31 supplies the needed names if they are not the default.” The RHEL6 kernel is 2.6.32 so I have revised the rules accordingly.

(Remember, if you use zaoe you have to modify the mknode setting in the script to suit your version of Red Hat linux. Currently the default is RHEL6, although it’s mostly being used in RHEL5.)
For young men, it is important viagra levitra online supplementprofessors.com to understand the cause of the dysfunction. Quite a lot of men are having less sex and therefore fewer babies with younger women. levitra vardenafil generic The doctor has two methods of determining pain during an examination levitra 20 mg for fibromyalgia, digital palpation and the use of illicit drugs. As per studies, herb purchase levitra online discover for more like tribulus terrestris is found to be very useful to increase the level of testosterone in the body.
Another, more worrisome issue is that using the Red Hat supplied kernel module causes problems with the new VSX and ESM hardware, but using the latest coraid module makes the kernel throw tons of ‘aoe: unknown ioctl 0x1’ errors. Damned if you do and if you don’t!

Red Hat just doesn’t get AOE

This post edited to correct my mistakes

ATA-over-Ethernet, the high performance low-cost SAN protocol developed by Sam Hopkins over at Coraid, never gets any love from Red Hat Enterprise Linux. The AoE kernel modules included with any given release are always egregiously out of date, and don’t even seem to be contemporary with the kernels they’ve been distributed with. If you want to use up-to-date drivers, you have to either wrap up coraid’s kernel module sources with DKMS and support a complete build chain, or create kernel-version independent modules, or else you’ll get system breakage from routine yum updates. Either way you have to muck about building RPMs so you won’t break package dependency and inventory tracking, and you have to set things up to get rid of the old Red Hat module each time you get a kernel update.

And despite the inclusion of an (elderly) aoe module in Red Hat, they don’t provide any officially blessed aoe-tools package. Coraid maintains a set of simple aoe management utilities, a remote console app for their aoe devices, and a throughput testing app, all free open source software. The basic aoe-tools package has been a part of Fedora for some time now, and lately coraid has been bundling the sources for the tools with the driver sources.

In my shop, we’ve been running more than 12 terabytes of AoE storage infrastructure on Red Hat EL 3, 4, and 5 for eight years or more now. Currently more than 150 TB in production. I had some pretty major problems with AoE on RHEL6, and openly blamed Red Hat’s out of date drivers for them, but those problems have been resolved. We had a bad VLAN trunk, basically, so it was really our fault (my sincere apologies to those I mistakenly accused).

LMS provides a powerful, systematic database capability managing all of your marketing technology across multiple media including Voice Broadcasting. http://pharma-bi.com/2009/03/making-time-for-the-future/ cheap cialis These medical conditions can also sometimes prove to cheapest sildenafil uk be harmful for you. The success arte of this medication is above 95% and thus is a purchase viagra very effective medication for the impotence issues in men. For instance, if you sense that your office visit was unpleasant or unproductive, you do have cialis super http://pharma-bi.com/2010/04/ are right to do so. You know, it’s always seemed to me that ATA-over-Ethernet should be a natural win for Red Hat. The simplicity of the protocol, when compared to Red Hat’s approved iSCSI, reminds me strongly of the value proposition that linux represented ten years ago when compared to proprietary unixes. You can build an AoE SAN that’s faster and more reliable than an iSCSI SAN for considerably less money; why would anyone purposely choose the less cost-effective solution? That’s why so many IT shops left HP-UX, SunOS, and MVS for Red Hat linux – because linux delivered the capabilities we needed for less cash.

Strangely, though, Red Hat treats AoE like an unloved and ugly stepchild, at best neglecting it entirely. Whenever a Fedora release is repackaged for publication as a Red Hat Enterprise Linux release, the aoe-tools package is removed; bleeding-edge Fedora probably provides a more stable AoE SAN platform than the flagship product. It’s deeply weird behavior and counterproductive for Red Hat.

I wrote an initscript for AoE under Red Hat that works under RHEL 3 through 6. It’s in the software section.

There’s an interesting discussion that references this blog post here.

udev under RHEL6 on a Dell m1000e blade server

The udev subsystem dynamically responds to udev events generated by the kernel and creates device nodes on the fly. This solves the old *nix problem of too many files in /dev that don’t have anything to do with the hardware you actually have. If you see a file named /dev/mouse, the system really truly has a mouse attached to it. Or at least that’s the plan. The kernel calls udev whenever you activate or deactivate any device, not just USB and PCMCIA cards, and this lets linux support absolutely any kind of hot-swappable hardware, even processors.

This is not the first dynamic hardware mapping scheme to hit linux – I think Red Hat EL4 used devfs and there was at least one other one before that. And people have tried to do it using the HAL daemon too.

The udev daemon reads “rules” that tell it how you want your devices named. This is so that different distributions that have historically used different names can all use udev, they just have their own rule sets. This also means that hardware invented tomorrow is trivial to add – just make a new rule.

In Red Hat Enterprise Linux v6, udev rules are read from multiple places in order to be maximally confusing and infuriating.

/lib/udev/rules.d is a “library” of standard rules that udev uses by default.

/etc/udev/rules.d is a place where you can put your own system-specific rules that will override the library. This is based on the names of the files, not the rules themselves.

/dev/.udev/rules.d is a secret hidden directory that seems to exist mostly to make you angry. I haven’t found any clear documentation of precedence of rule files found here. You have to experiment on a test system. The rules in here are usually referred to as “temporary rules” which may very well mean “permanent overrides” in normal English.

udev automatically detects changes to rules files, so changes take effect immediately without requiring udev to be restarted. However, the rules are not re-triggered automatically on already existing devices, so you might have learn to use the udevadm command (or just reboot) if you write any new rules.

But wait! There’s more! It was starting to make sense, and we can’t have that!

If you have a Dell system, a program called biosdevname will be invoked by rules in the library file /lib/udev/rules.d/71-biosdevname.rules whenever the system becomes aware that a network interface exists. The Dell white papers on the subject tell me that this program will rename your network devices and disk drives in a way that’s both counter-intuitive and inaccurately documented, but in reality it just renames the ethernet interfaces on a specific set of Dell systems.

Now, honestly, on some models of Dell server this behaviour makes a decent amount of sense; it labels the embedded ethernet ports on the system’s motherboard as “em1” and “em2” which is a good idea since Dell labeled the ports 1 & 2 (instead of the expected 0 and 1 – Real Computer Scientists can’t count past nine on their fingers). Then it will label any additional interfaces by the PCI bus positions, which is sort of uselessly informative but at least consistent.

On a Dell M1000e blade server, though, running biosdevname is worse than doing nothing. Instead of following their reasonably sane idea of making the labels on the chassis match the labels on the interfaces, the emn and pxpy naming scheme is mapped pseudo-randomly across the network ports. Net fabric A is arbitrarily designated as “embedded” (remember, on the M1000e none of the interfaces are embedded in the blades themselves), P3 is considered fabric B, and P1 is assigned to fabric C – even the order doesn’t match. So, on blade five, the interface names end up looking like this:

This generic cheap viagra circulation controls blood flow into the male sex organ and cause an erection. Men can feel much better erection if they neglect eating grapefruit or avoid drinking side effects cialis grape juice. This medication is available to you purchasing viagra in two low dosage strengths of 2.5mg and 5mg. If overmuch skin is taken away, or a more upright vector not employed, the order levitra human face can assume a pulled-back, “windswept” visual aspect. Blade Chassis Name -> Linux Interface Name
A1-port5 -> em1
A2-port5 -> em2
B1-port5 -> p3p1
B2-port5 -> p3p2
C1-port5 -> p1p1
C2-port5 -> p1p2

This is like purposeful obfuscation of location, and much worse than just nailing down eth0 through eth6 to specific MAC addresses (which RHEL6 would happily do if Dell’s biosdevname software wasn’t sticking its member into the pie).

The best solution I’ve found so far is to create a file named /etc/udev/rules.d/71-biosdevname.rules (thus overriding /lib/udev/rules.d/71-biosdevname.rules) and put a series of udev rules in it that associate specific names with specific MAC addresses. I map the actual Dell chassis labels like so:

Blade Chassis Name -> Linux Interface Name
A1-port5 -> ethA1p5
A2-port5 -> ethA2p5
B1-port5 -> ethB1p5
B2-port5 -> ethB2p5
C1-port5 -> aoeC1p5
C2-port5 -> aoeC2p5

With this setup, you will know which wire the telco guy unplugged as soon the system starts screaming about something gone wrong on ethA1p5 – it’s the Cat 6 ethernet wire in slot A1 port 5, obviously. You can also see which interfaces are intended to be used for our high-speed ATA-over-Ethernet SAN infrastructure.

That’s enough for today…