It’s always fun playing around with technology, and we do like to find any excuse to put our 3D printer to good use. So when our resident 3D guru Chris found online the design for a Lego toilet roll holder we just had to give him lots of encouragement. Not least because it is considerably more attractive than the Space Invader which graces our downstairs staff loo.
Then we found a spare Apple sticker kicking around. I’m sure that when Apple throw stickers in with their devices they hope you will stick them in your car window (so long as you drive an executive model), or on your designer briefcase. But we’ve always taken great pleasure in using them to disguise our laptops, so we can infiltrate Apple-only buildings with cunningly concealed Dell gear. This lets us appear to be equally hobbled by the Fisher Price interfaces that technophobes love but real programmerscan’t stand and yet strangely be more productive than the “programmers” who prefer to work on Apple devices. Anyway, all our Windows laptops already have Apple stickers, so the spare became a T-Shirt logo, and very nice it looks too…
Please note: for the hard of humour, the paragraph above was a deliberate wind-up, we’re actually fairly OS-ambivalent at SteamDesk, we work on all sorts of platforms here.
We’re very pleased to announce that SteamDesk was shortlisted (one of just four companies) for the Innovation and Tech Awards 2019 in the “Innovative Product of the Year”. Our submission was the Acorn Dash client, job and time management system which we introduced earlier this year and is currently in public Beta test. If you are interested in a residual income for life, then please get in touch, we are looking for affiliate marketers who will help us to promote the product in return for generous ongoing commissions.
We were kindly invited out earlier this week to join the Eledecks team for a Christmas dinner (www.eledecks.com). We have been working for them for nearly 6 years now and have watched their business go from strength to strength. Eledecks provides “white label” HR services to companies via a network of resellers, and the back-end programming has been very interesting with a lot of variety. Because we came together with them after they had already been established for a number of years we have had to deal with a substantial amount of legacy systems support. Juggling their constant desire to add new features to their system with our goal to improve the existing code while always ensuring security and quality has taken some fine balancing, and we’re very proud of the work we have done for them. They are also a lovely team to work with, so thanks again Carolyn and her team for taking us out, and long may we continue to work together!
Eledecks is one of the many clients that we have picked up because they have chosen to part company with their previous developers. If you are fed up with developers who try to dazzle you with technobabble, who charge a lot for doing very little, and who dodge your calls or prefer to point fingers rather than solve problems, then perhaps your New Year’s Resolution could be to talk to us too. Life is too short to stick with bad suppliers, and the short term pain of a sudden switch is often better than the “death by a thousand cuts” of sticking with the devil you know. If this sounds like your circumstances, we would love to hear from you.
The Drupal developers have a lovely habit of using the word “regression” in their version notes to politely refer to fixing bugs that were introduced in previous versions. So where Linus Torvalds might famously record “I fixed this ****ing idiot’s code”, the more genteel Drupal community will record a phrase such as “Fixed regression in the link widget where help text does not show”. It all boils down to the same thing though – going back to fix something that somebody just broke.
My question was, is this a case of “two steps forward, one step back”? Fortunately I could see a simple way to quantify things. All of the release notes are listed on pages starting from here: https://www.drupal.org/project/drupal/releases … so with a quick bit of command line coding, I was able to knock together a few data trawls so that I could then count the number of times the phrase has been used in updates to the last three versions of Drupal – 8, 7 and 6.
The code for Drupal 6 was easy – there were 38 revisions posted for this major version, and the development team was very consistent about numbering them. These few bash commands were all I needed to pull down the release notes and save all of the lines mentioning “regression” while filtering out the few that said “no known regressions”.
for start in {0..38}
do
curl -s "https://www.drupal.org/project/drupal/releases/6.$start" | grep regression | grep -v "no known regressions" >> drupal_6_regression_mentions
done
I will save the actual figures for the end of the article.
The code for Drupal 7 was very similar, except there were 57 revisions for that version:
for start in {0..57}
do
curl -s "https://www.drupal.org/project/drupal/releases/7.$start" | grep regression | grep -v "no known regressions" >> drupal_7_regression_mentions
done
For Drupal 8, the current version, they have moved the goalposts a bit – revision numbers now have major and minor parts, and scattered in amongst them are “dev”, “alpha”, “beta” and “rc” versions. In the interest of fairness I decided to ignore all of those. To do this, I looped across all of the pages of the Drupal 8 releases, created a list of all the links to release note pages that didn’t use those phrases in their names, and then looped across that list to pull down each page and then look for my keyword, “regression”. That resulted in a list of 38 separate revisions so far in Drupal 8, and my code looks like this:
for start in {0..4}
do
curl -s "https://www.drupal.org/project/drupal/releases?api_version%5B0%5D=7234&page=$start" | grep "/project/drupal/releases/" | egrep -v "(beta|alpha|rc|dev)" | sed -e "s:.*href=\"\([^\"]*\)\".*:\1:g" >> drupal_8_notes.tmp
done
sed -e 's#https://www.drupal.org##g' < drupal_8_notes.tmp | sort | uniq > drupal_8_notes
while read url
do
curl -s "https://www.drupal.org${url}" | grep regression | grep -v "no known regressions" >> drupal_8_regression_mentions
done < drupal_8_notes
So now for the fun part, the figures. I’ll lay them out in table form and then give a little commentary:
Drupal Version
Number of Revisions
Number of Regressions
Ratio of Regressions to Progressions
6
38
5
7 and 1/2
7
57
37
1 and 1/2
8
39
17
2 and 1/2
The last figure gives the number of “steps forward” that are taken before each single step back, with the figures rounded to the nearest “half step” (or “stumble” as I prefer to think of them) .
So what do we see? Well, Drupal 6 looks like it was really successful, with 38 steps forward and only 5 steps back. Of course, those of us who used it in anger know this definitely was not the case (D6 was, in technical terms, a pig!), so presumably they simply did not use the term “regression” until late in its life. Perhaps someone else would care to investigate further…
Drupal 7 was to all intents and purposes still a bit of a mess – with only 1 and 1/2 steps forward for each step back. It certainly did sometimes feel like that at the time – we apply each of these patches to our clients’ sites as they arrive, and with the best will in the world Drupal is fundamentally a community-led project and there is not all that much real accountability. Getting Drupal developers to agree on how something should be done is like herding cats. So errors and contradictions do often slip through. In essence, some of what you gain in the short term due to the flexibility and low cost of the Drupal platform you go on to lose in the longer term because of its greater maintenance requirements. This harks back to my previous blog – I would still never recommend Drupal or Magento to small companies because their maintenance requirements are just too high. If you compare these two systems with WordPress for instance, the update cycle and mechanism seems archaic. Modern WordPress installations auto-update and they very rarely fail if they have been built on popular, standard plugins. Magento and Drupal still require a lot of ongoing developer support. While companies like SteamDesk are happy to make a living providing this, we really feel for the smaller clients who have been pushed down the Magento/Drupal route by inexperienced developers in the past when it really was not the best choice for their requirements.
Coming back to the story, we finish with Drupal 8, which certainly appears to have introduced a reasonable improvement over Drupal 7. There are now 2 and 1/2 steps forward for each step back, which is still very close to the apocryphal “two steps forward, one step back” but it is still moving in the right direction. So apparently things are currently looking up in the Drupal community, hooray!
Caveat: the figures above are a bit silly really, because a new version is very rarely a “single step forward” (a new version usually includes multiple, sometimes even dozens of improvements), while what I am counting as a regression is typically a “single step backward”. So the actual ration of regression to progress is going to be much lower than I’ve shown above. But that wouldn’t have made for such a fun story!
We were recently approached to help a client with their Magento website, and as a matter of course we first started with a check of what version they were running, what their current patch status was, and so on. We were immediately alerted to the fact that the site had not been patched for some time, and had recently been hacked. There’s a great online tool to help with this at MageReport.com:
It was showing lots of red marks on the report, most of which were warnings that patches haven’t been installed, but our eyes were drawn to the worst, the one for “Credit Card Hijack detected?” which actually meant that the site wasn’t just vulnerable, it was already compromised. This was easily confirmed by looking at the source code for the home page. At the end of it, just before the closing </html> tag there’s an obfuscated bit of JavaScript (please excuse the ham-fisted redacting of the actual domain name)…
I could easily work out what the script does by using the debugging console in my browser (I’m using Chrome here but you can do it in pretty much all the browsers). Copy the Javascript, all except for the ‘eval’ command at the end…
Then paste it into the JavaScript debug console in the (F12) developer tools:
Then for this particular script when you hit return it shows you what the variable x evaluated to (variations on this technique pretty much always work, although you do have to know what you are doing to avoid triggering some malware by mistake):
We’ve seen very similar code before, our assumption is that this is getting dropped on as part of a scripted “drive-by” attack. If this succeeds the script presumably keeps a note that your site is vulnerable and moves on to the next one. Some time later, the hackers visit again and drop back doors all over the place (we found more with just a cursory glance around the site).
In the screen grab above you can see the “send” function posts the form data to a destination page which has been named to sound like an innocuous jQuery library download ( trafficanalyzer.biz/lib/jquery-1.9.1.min.php ) but is actually isn’t that at all but is a script to capture the stolen data (the .php is another good clue, jQuery is a .js library file usually). I haven’t got the whole script in view in the screenshot above, there is a bit which has scrolled off the top which limits it to only trigger when you are on the site’s checkout/cart pages. To add insult to injury, there’s also a simple bit of code there which double checks whether any credit card numbers it is stealing look valid, using a simple regexp check. If it thinks it might have found a good credit card number it alters the destination URL slightly. This makes me laugh a bit to be honest, it is so cheeky of the hacker to use the victim’s processing capacity when it would have been trivial to do this check at the reverse end. They really do seem to be rubbing the victim’s nose in it.
You know that the hacker must have acquired admin level access into Magento in order to have dropped this hack in place. They have also turned on the on-site Sage Pay mechanism, so that rather than the two off-site ones which the client *thought* were enabled there were now three options when you got to the checkout step:
The third option has been turned on because the other two would take the user off-site to get credit card details. This is no good for the hacker (their script stops working when you go off to the remote payment gateway). By keeping you on-site they ensure that they will definitely get your credit card details as well as the personal details (address, phone number, email, etc) that they would get whichever payment mechanism you used.
This hack has certainly happened due to missing patches and/or poor password security. You might think you just need to clean up the site, indeed there are tips at this web address for how to remove the script via the Magento control panel:
…. (plus of course, you should remove the on-site payment mechanism). But our experience is that once someone has access to the Magento admin area they will have compromised the site in multiple ways and left themselves back doors – if there isn’t a clean copy of the site in source control to review against I would always strongly advise re-building rather than attempting repairs. Magento is a heavy beast (30,000 files plus in a typical install), the best programmer in the world couldn’t sensibly check all of those for hacks without the appropriate reference copies to check against, which means having the original versions of all of the plugins/extensions that the site uses. Without source control the chances of getting the right versions of all those files and making full comparisons in anything like a sensible timescale are absolutely zero. I wrote a blog entry detailing our processes a year or so ago:
It gets worse… In this particular instance we were quickly able to see that the hacker had also dropped a similar script into the “protoype.js” library which all site pages used (even the admin areas), so it was clear that if someone just followed the “how to fix credit card hijack” instructions above they would have immediately leaked their new admin passwords to the hacker as soon as they next logged in.
Of course, the real solution is to put a company like SteamDesk on retainer (our packages start from 4 hours/month), and we will monitor your site, update it straight away whenever new security patches are released, and add source control/backup mechanisms so that we can *recover* a site after it has been damaged (because with the best will in the world, you might still suffer from a zero-day attack, or a malicious insider, or a member of staff exercising poor password control) rather than have to start all over again.
We’re just playing with If This Then That ( https://ifttt.com )
We are working on a new AI project, to monitor things and give you warnings when it might be going wrong. If This Then That is a great web service which lets you connect things together, we’re demoing it to someone by making it send a text message when a new blog post appears.
So this is the blog post.
And here’s a gratuitous picture of everyone at SteamDesk, to make it more interesting.
You must be logged in to post a comment.