Our Blog
Solutions. Made Simple.

Forced upgrade from Centos 6.8 to Centos 7.2


Last Friday I made the foolish mistake of getting bored with the endless sequence of manually updating our development servers with operating system and software patches. In a moment of wishful thinking, I decided that since the manual process hadn’t caused any problems in years, I’d take the rash step of automating it and relying on yum-cron to keep our proxy server up to date. Oh, what a good idea that proved to be.

Monday morning, and the office network is fairly upset, largely down to the proxy VM sitting with an unresponsive screen with just a lovely “Kernel Panic!” message. Haven’t seen one of those for a while, but it’s like getting the Blue Screen of Death or the Apple Bomb, you know it isn’t going to be a good day.

An hour or so later, I’ve abandoned the hope that loading an older kernel (which still worked) was going to get me to the cause of the problem, which seemed to be related to selinux (which was, in theory, disabled anyway). Documentation online was patchy – this is a fairly old OS, so most of the advice is to upgrade. Rather than waste all day fixing something which wasn’t worth that to me, I thought I’d either upgrade it or run up a new virtual machine. I decided on the former, even though prevailing opinion was that C6->C7 upgrading isn’t recommended, because I thought our minimal server fell into the “nothing unusual about the configuration” category that just might work. How wrong could I be?!

How wrong indeed… here’s what happened, with the more tedious stuff redacted. Basically the official upgrade process is – well, pants is an understatement, and I don’t like to swear in this blog, so I’ll just say… rubbish! It leaves behind no end of version 6 files that conflict with the new version 7 ones, or go missing, or refuse to update, or refuse not to update. You name it, if it can go wrong it will!

Geek stuff below. The end result – in parallel with this I did the fresh install solution too, and in a quarter of the time that I had wasted on trying to go this route I had a fresh C7.2 VM up and running and doing all the proxy/dns things that the old one did. Now, the question is, should I install yum-cron… ?!

[[email protected] ~]# yum update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.clouvider.net
* epel: epel.check-update.co.uk
* extras: centos.serverspace.co.uk
* updates: centos.serverspace.co.uk
Resolving Dependencies
--> Running transaction check
---> Package apache-commons-collections.noarch 0:3.2.1-21.el7 will be updated
---> Package apache-commons-collections.noarch 0:3.2.1-22.el7_2 will be an update
---> Package apache-commons-discovery.noarch 2:0.5-9.el7 will be obsoleting
---> Package avahi-glib.x86_64 0:0.6.31-15.el7 will be updated

[... and more, very boring ...]

[... Wait a minute, what have we here? Centos 6 packages still? ...]

---> Package python-webhelpers.noarch 0:0.6.4-4.el6 will be updated
---> Package python-zope-sqlalchemy.noarch 0:0.4-3.el6 will be updated

[... and more, very boring ...]

---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed
---> Package python2-ecdsa.noarch 0:0.13-4.el7 will be installed
--> Running transaction check
---> Package libtommath.x86_64 0:0.42.0-4.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================================================================
Installing:
apache-commons-discovery noarch 2:0.5-9.el7 epel 81 k
replacing jakarta-commons-discovery.noarch 1:0.4-5.4.el6
kernel x86_64 3.10.0-327.36.1.el7 updates 33 M
python2-crypto x86_64 2.6.1-9.el7 epel 475 k

[... and more, still very boring ...]

perl-Linux-Pid x86_64 0.04-18.el7 epel 14 k
python-cherrypy noarch 3.2.2-4.el7 base 422 k
python-repoze-lru noarch 0.4-3.el7 epel 13 k
python2-ecdsa noarch 0.13-4.el7 epel 83 k

Transaction Summary
==============================================================================================================================================================================================================
Install 5 Packages (+7 Dependent packages)
Upgrade 190 Packages

Total download size: 263 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/202): avahi-glib-0.6.31-15.el7_2.1.x86_64.rpm | 24 kB 00:00:00
(2/202): avahi-libs-0.6.31-15.el7_2.1.x86_64.rpm | 61 kB 00:00:00
(3/202): apache-commons-collections-3.2.1-22.el7_2.noarch.rpm | 509 kB 00:00:00
(4/202): bind-license-9.9.4-29.el7_2.4.noarch.rpm | 82 kB 00:00:00

[... and more, still very, very boring ...]

(198/202): wireless-tools-29-13.el7.x86_64.rpm | 103 kB 00:00:00
(199/202): util-linux-2.23.2-26.el7_2.3.x86_64.rpm | 1.9 MB 00:00:00
(200/202): selinux-policy-targeted-3.13.1-60.el7_2.9.noarch.rpm | 3.9 MB 00:00:02
(201/202): qt-x11-4.8.5-12.el7_2.x86_64.rpm | 13 MB 00:00:05
grub2-tools-2.02-0.34.el7.cent FAILED 99% [============================================================================= ] 3.0 MB/s | 261 MB 00:00:00 ETA
http://centos.hyve.com/7.2.1511/updates/x86_64/Packages/grub2-tools-2.02-0.34.el7.centos.x86_64.rpm: [Errno 12] Timeout on http://centos.hyve.com/7.2.1511/updates/x86_64/Packages/grub2-tools-2.02-0.34.el7.centos.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
Trying other mirror.
(202/202): grub2-tools-2.02-0.34.el7.centos.x86_64.rpm | 3.3 MB 00:00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 6.7 MB/s | 263 MB 00:00:39
Running transaction check
ERROR with transaction check vs depsolve:
python(abi) = 2.6 is needed by (installed) python-repoze-who-testutil-1.0-0.4.rc1.el6.noarch
python(abi) = 2.6 is needed by (installed) python-repoze-what-1.0.8-6.el6.noarch
python(abi) = 2.6 is needed by (installed) python-repoze-what-pylons-1.0-4.el6.noarch

[... and more, very boring ...]

httpd-mmn = 20051115 is needed by (installed) mod_extract_forwarded-2.0.2-8.el6.x86_64
** Found 75 pre-existing rpmdb problem(s), 'yum check' output follows:
classpathx-jaf-1.0-15.4.el6.x86_64 has missing requires of java-gcj-compat
cloog-ppl-0.15.7-1.2.el6.x86_64 has missing requires of libgmp.so.3()(64bit)
freetds-0.91-2.el6.x86_64 has missing requires of libgnutls.so.26()(64bit)
freetds-0.91-2.el6.x86_64 has missing requires of libgnutls.so.26(GNUTLS_1_4)(64bit)
grep-2.20-5.el6_8.x86_64 has missing requires of libpcre.so.0()(64bit)
hal-info-20090716-5.el6.noarch has missing requires of hal >= ('0', '0.5.10', None)
jakarta-commons-pool-1.3-12.7.el6.x86_64 has missing requires of java-gcj-compat
libgcj-4.4.7-17.el6.x86_64 has missing requires of libgmp.so.3()(64bit)
matahari-lib-0.6.0-14.el6.x86_64 has missing requires of libpcre.so.0()(64bit)
matahari-lib-0.6.0-14.el6.x86_64 has missing requires of libpython2.6.so.1.0()(64bit)
mod_extract_forwarded-2.0.2-8.el6.x86_64 has missing requires of httpd-mmn = ('0', '20051115', None)
mod_perl-2.0.4-11.el6_5.x86_64 has missing requires of httpd-mmn = ('0', '20051115', None)
mod_perl-2.0.4-11.el6_5.x86_64 has missing requires of perl(:MODULE_COMPAT_5.10.1)

[blah blah blah]

Your transaction was saved, rerun it with:
yum load-transaction /tmp/yum_save_tx.2016-10-10.16-12.CZ4VLB.yumtx

How to prevent your staff from using iPlayer


…or, “No, I will not be buying a BBC TV licence for my staff”

Well, the law changed yesterday so that allowing people to use the BBC iPlayer at work makes their company liable to pay for an annual TV licence. What a rip-off! So you’re now forcing people who pretty much universally pay for a TV licence at home to have to pay for another TV licence to cover their staff, who also pretty much universally pay for a TV licence at home. I’ve yet to see a more clear indication that the fools who make decisions like this for the government and the BBC should really be unemployed. If anyone has ever wondered what has happened to the traditional “village idiots”… I can show you where they are.

Well, I’m not standing for this. The BBC is a poor enough service as it is, wasteful, expensive, biased, you name it, it’s easy to criticise. It wouldn’t bother me, except I’m expected to directly pay for it too. There’s no opting out. Do you know how many BBC TV programmes I have watched in the last few months? Two or three half-episodes of Robot Wars and Bake-Off. I watch good quality programmes on Netflix most nights, for £7/month. I pay the BBC more than twice as much, and watch it maybe 2% of the time. I do admit to reading the BBC news website a lot though, so at least they do something well, but given the choice between paying £145.50 a year or going without any BBC services, I know which I would pick.

Anyway, in anticipation of the constant stream of threatening letters from the TV Licensing Authority, I’ve just made a few simple router configuration changes which prevent my staff from accessing iPlayer. These changes, documented here for posterity (and date-stamped proof) use our Draytek Router’s features to block specific URLs, which I’ve set up to catch all the variations of iPlayer that I can identify. I’ve also told my staff that the company rules dictate that they not use iPlayer at work. I think this is reasonable and sufficient behaviour – if the BBC chooses to move the goalposts without telling me (e.g. moving the address of iPlayer) then I’ll deal with that as and when I notice.

So here’s a set of useful screen grabs which show you how to do it…  I think you probably don’t need to enable the “DNS Filter” setting actually.  Turning that on gave us some issues, and after turning it off we’re currently locked out of the router admin interface, it’s looking like I might end up having to recover it from the latest backup and re-apply the other changes.

This slideshow requires JavaScript.

[edited: 5th Sept, to fix the misspellings. Also it’s worth noting I can now confirm, *all* my staff have full TV licenses at their home addresses]

UCL ENGins site launch


The culmination of a lot of work over the summer came this week, with the launch of a new website for the UCL engineering department.

The site, uclengins.org is a  heavily customised WordPress platform with an advanced tagging mechanism and has “gamification” features which encourage site users to become site contributors and work their way up the leaderboards.

UCL ENGins

We inherited an “interesting” code base to build from and bring this site to life, which proved quite a challenge to deal with when WordPress 4.2/4.3 arrived (with its major changes to the way taxonomy terms are handled) mid-way through the project and our decision to stay up to date resulted in some sleepless nights…  Hats off to Dave Langley for his sterling work getting on top of it in time for the big launch earlier this week!

New Bjorn Borg range at our www.pants2you.com website


As you probably already know, we’re great with Magento at SteamDesk. We run a little “real-world” Magento website as a test platform, to make sure that we’re always on top of patching issues, and to give us a location to try out interesting extensions. We also syndicate out to eBay and Amazon using m2ePro. I can’t imagine anyone buying Magento development or support from a company that doesn’t also have experience as a product user.

Anyway, this week we’ve updated our stock from one of our favourite suppliers, Bjorn Borg, so that our site has lots of variety in the run up to Christmas. The guys in the office are snaffling “review pairs” of these pants at such a rate that we might end up having to order again, just to carry enough stock!

Three packs are great value for money, and bizarrely also provide us with the best mark-up…

Triple Packs from the Bjorn Borg Winter 2015 range

Triple Packs from the Bjorn Borg Winter 2015 range

But our favourite designs are all in the singles – the Shaman pair is almost as good as the 8-bit joystick pair that came out last year…

Single and Double Packs

Single and Double Packs

New SteamDesk site design – summer 2015


We’re pleased to launch a new site design for the summer of 2015. This is based on design sketches provided by Wayne Christian which he kindly made for us while working with Studio View, he can now be found at A Digital Engagement.

The site has been put together by our Summer Intern, Hannah Elias, who stayed with us for nearly 4 months over the summer while on vacation between the second and third years of her degree at Northumbria University. Thanks Hannah, you’ve done a lovely job. While here, Hannah also worked on a major data forms project for Osiris Educational, OTI Forms (sadly you can’t see much of this without a login), re-worked the templating in William Reviews to be mobile-friendly and responsive, and worked extensively on The Best Of Staffroom, a website showcasing educational articles. Best of luck this year, Hannah!

hannah

Speeding Up Magento With Redis


We recently developed an online shop in Magento and were looking at ways to improve the site’s responsiveness. Magento can be a lumbering beast at times, with it’s endless feature list and huge number of database tables, so performance tweaking is essential if you want to build a successful e-commerce platform.

Having worked on a number of Magento shops before we ran through the usual optimisation techniques (enabling caching, enabling compilation, using Apache’s GZIP compression to compress pages and setting up reasonable caching times for resources) but we were still hungry for more speed. Our research then lead us to Redis, an open source, in-memory, key-value store which can be used to replace the filesystem based cache built into Magento.

So, we knew we wanted to use Redis… What next?! Some more research threw up the key points which will be important to anyone wanting to implement Redis with Magento.

Firstly, Magento CE 1.8 and up comes with the required modules built in and is therefore much easier to get up and running. The modules do exist for versions 1.7 and below, so it’s still possible, but it does involve more work and it could simply be easier to upgrade Magento to the latest version (1.9.0.1 at the time of writing). Forgive the shameless plug but if you do need help upgrading Magento we have experience all the way back to 1.3 so get in touch.

Secondly there are a number of prerequisites you’ll need to take care of before configuring Magento. Specifically you’ll need the following, latest versions are always preferable:

  • Redis server (naturally!)
  • PHP Redis extension

And that’s about it! The above can be installed on most Linux systems using yum, as follows:

yum install redis php-pecl-redis

And you should be left with a system that’s ready to go. First off we need to start the Redis server, make sure it comes back if the server is ever restarted and check it’s running:

/etc/init.d/redis start
chkconfig redis on
redis-cli ping

And reply of ‘PONG’ means everything is looking good. We also need to restart Apache and check the Redis support we just installed is loaded:

/etc/init.d/httpd restart
php -m | grep redis

You should see a single word response from the last command, ‘redis’, which confirms the module is ready to go. Next we need to modify the Magento local.xml file with details of our new Redis server and what we want to use it for. Open up app/etc/local.xml and add the following anywhere in the <config><global>…</global></config> block:

<redis_session>

<host>127.0.0.1</host>

<port>6379</port>

<persistent>sess-db1</persistent>

<db>1</db>

<compression_threshold>2048</compression_threshold>

</redis_session>

<cache>

<backend>Cm_Cache_Backend_Redis</backend>

<backend_options>

<server>127.0.0.1</server>

<port>6379</port>

<persistent>cache-db1</persistent>

<database>2</database>

<compress_data>1</compress_data>

<compress_tags>1</compress_tags>

<compress_threshold>20480</compress_threshold>

</backend_options>

</cache>

All of these are basically defaults, with the exception of the <database>/<db> and <persistent> settings. These need to be set uniquely for the server otherwise you will most likely get weirdness of all kinds which is never a good thing. At this point you’ll also want to make sure the <session_save> option already in your local.xml is set to ‘db’ as this is overwritten at runtime by the Redis module.

Next step? There isn’t one! Magento will now use the Redis server to store user sessions and page cache, which should see your page response times cut in half. We had some basic benchmarks and we saw a 25%-50% reduction in page response times across the site, even including pages which wouldn’t normally receive a boost from caching (such as the checkout). Maybe this reduced the server load sufficiently that it appeared to speed up these pages despite not having a direct effect? We haven’t looked into it enough yet to know for sure but we do know the site is now running great and using less resources than we’ve ever seen Magento use before.