Tag Archives: security

WordPress connectivity issues (Jetpack and XML-RPC failure)

Summary: Some WordPress features can be abused and are therefore locked down by hosting companies. My hosting company recently made changes to their security which broke features I use for my blog. The fix, once identified, was quick and easy.

I’ve been blogging for around five years and am impressed with how easy and reliable WordPress has been over that time, despite constant updates. Earlier this week however I logged into my WordPress console and was greeted by an innocuous looking error message;

Wordpress error

 

As suggested I tried disconnecting Jetpack and reconnecting but that didn’t work – Jetpack refused to reconnect and gave an error message saying my site wasn’t publicly accessible;

Wordpress error2

I tested the site which seemed to be available and working as expected. Diving a bit deeper I read into the plugin connectivity requirements and found that Jetpack (among others) relies on the XML-RPC protocol, which is now enabled by default in WordPress since v3.5. At a basic level you can test this by putting a simple URL in a browser – http://yourWordpressSite/xmlrpc.php – and it’ll return the single line ‘XML-RPC server accepts POST requests only‘. This worked fine for me but knowing that the mobile (iOS and Android) WordPress app used XML-RPC I tried those and found they weren’t working. Hmm.

At this point I logged a call with WordPress (and generated a debug bundle) who were quick to confirm that from their perspective the website wasn’t responding to a ‘curl’ request (different to the basic test above) and they advised talking to my hosting company. I’d recently upgraded WordPress to v4.2 so suspected that might be related and in my opinion it seemed unlikely that the hosting company had locked down such a popular feature, certainly without notification, but I logged a call with them just to be sure. I was wrong and WordPress support were right! My hosting company (EvoHosting, highly recommended) advised me that due to DDoS attacks using aspects of the XML-RPC functionality they’d been forced to restrict it.

The fix was to install an additional WordPress plugin which limits the XML-RPC functionality (it stops XML-RPC pingbacks) but still allows the more popular features typically used by mobile WordPress apps and some plugins like Jetpack. With this installed they were able to whitelist my site and I was able to reconnect Jetpack and get my mobile apps working again. Obviously this fix won’t work for everyone as it depends on how restrictive your hosting company are – they may block XML-RPC completely, in which case you’ll have to plead your case. WordPress have a list of recommended hosting companies who all allow allow this functionality.

NOTE: I also believe this was the root cause of my the Jetpack Publicize issue whereby LinkedIn ‘needed refreshing’ constantly. Two birds with one stone…

Morale of the story – just because you work in IT don’t assume you know more than support teams. Some of them are very good and know their stuff! Guilty. 🙂

Further Reading

WordPress XML-RPC PingBack Vulnerability Analysis

BetterWPSecurity – a great WordPress plugin but proceed with caution

I’ve recently installed the BetterWPSecurity WordPress plugin, and found that while it’s very useful and does increase the security of WordPress it can also break your site.

Ah, Monday morning and the start of my three months paternity leave looking after my six month old son Zach. During his morning nap I logged into my blog to work on an article and noticed that my blog wasn’t loading articles correctly even though the home page worked just fine. Investigating further and looking at my site stats (I use both the Jetpack plugin and Google Analytics) clearly showed that something broke at the start of the weekend – I had nearly no traffic all weekend. Having just referred a colleague to my site for some information and on my first day of paternity leave (ie less time on my hands, not more as some may think) this was definitely not ideal timing!

My first step was to check my logs for information, in this case the BetterWPSecurity log for changed files. This revealed that the .htaccess file in the root directory was changed late on Friday night at 11:35pm – and I knew that wasn’t me as I was tucked up in bed. My first thought was a hack as the .htaccess file permits access to the site but there was no redirect or site graffiti and the homepage still worked so that didn’t seem likely. I logged in via SSH to have a look at the .htaccess file but didn’t see anything obvious although I’m no WordPress expert.


My priority was to get the blog working again so I tried restoring a copy of the changed file from the previous week’s backup (made via the BackWPUp plugin) only to find the backup wasn’t useable. Bad plugin! Luckily I’m a believer in ‘belt and braces’ and I knew my hosting company, EvoHosting, also took backups. I logged a call with them and within the hour they’d replied with the contents of the file from a week earlier. Sure enough the file had been changed but looking at the syntax it appeared to be an error rather than malicious hack.

My .htaccess file when the site was working;

# BEGIN WordPress

RewriteEngine On

RewriteBase /

RewriteRule ^index\.php$ - [L]

RewriteCond %{REQUEST_FILENAME} !-f

RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule . /index.php [L]

# END WordPress

My .htaccess file after the suspicious change;

# BEGIN Better WP Security

Order allow,deny

Allow from all

Deny from 88.227.227.32

# END Better WP Security

RewriteBase /

RewriteRule ^index\.php$ - [L]

RewriteCond %{REQUEST_FILENAME} !-f

RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule . /index.php [L]

</IfModule>

# END WordPress

I backed up the suspicious copy of the file (for future reference, ie writing this blogpost), restored the original et voila – the blog was working again. Step one complete, now to find the root cause…

Part of any diagnostic process is the question ‘what’s changed?’ and I had a suspicion that BetterWPSecurity could be the culprit as I’d only installed it a few weeks earlier. There was also the obvious issue of the new code in the .htaccess file which looked to belong to BetterWPSecurity. I checked the site access logs which confirmed my hypothesis – someone had attempted to break into my site and while attempting to block the attacker BetterWPSecurity had mangled my .htaccess file. The logs below have been truncated to remove many of the brute force login attempts (there were plenty more) but note that on the final line (after BetterWPSecurity has blocked the attacker) the HTML return code was 418 (“I’m a teapot”) rather than 200 plus the suspect IP 88.227.227.32 is the same as the one denied in the mangled .htaccess file. Yes, you read that right, “I’m a teapot”! Here’s a full explanation for that April Fool’s error code. 🙂

88.227.227.32 - - [15/Feb/2013:23:35:19 +0000] "POST /wp-login.php HTTP/1.1" 200 3017 "http://www.vexperienced.co.uk//wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"
88.227.227.32 - - [15/Feb/2013:23:35:19 +0000] "POST /wp-login.php HTTP/1.1" 200 3017 "http://www.vexperienced.co.uk//wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"
88.227.227.32 - - [15/Feb/2013:23:35:19 +0000] "POST /wp-login.php HTTP/1.1" 200 3017 "http://www.vexperienced.co.uk//wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"
88.227.227.32 - - [15/Feb/2013:23:35:19 +0000] "POST /wp-login.php HTTP/1.1" 200 3017 "http://www.vexperienced.co.uk//wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"
88.227.227.32 - - [15/Feb/2013:23:35:19 +0000] "POST /wp-login.php HTTP/1.1" 418 5 "http://www.vexperienced.co.uk//wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"

So BetterWPSecurity led me to the fault but also caused it. To be fair the plugin does warn you which settings are potentially going to cause issues but I’d assumed that it wouldn’t be me – dangerous things assumptions. I’ve rectified the issue by restricing BetterWPSecurity from altering core system files as shown in the screenshot below;

My blog is fixed and I’m feeling quite chuffed that it was all resolved during a long lunchbreak – not a bad day’s work if I do say so myself! Lesson for today? Take warnings seriously and have multiple backups!

VCAP-DCA Study notes–7.3 vShield Zones

vShield Zones is basically a firewall framework to protect your VMs without requiring external or hardware based firewalls. It requires Advanced or higher licencing. For study I’d suggest going through Eric Siebert’s blogposts (part one, two, and three) to start with (they cover real world issues) and then getting stuck into the official docs – they cover everything on the blueprint. There’s quite a bit to learn making this is one of the larger objectives on the VCAP-DCA blueprint.

NOTE: vShield Zones is NOT the same as vShield App, Edge, and Endpoint so make sure you download the right version. The VCAP-DCA exam only covers v1.0 of vShield Zones (not the most recent v4.1) and doesn’t cover the more feature rich vShield App Suite. See VMware’s product page for more details.

Knowledge

  • Identify vShield Zones components
  • Identify the four CLI command modes

Skills and Abilities

  • Configure vShield Zones
  • Backup and restore vShield Manager Data
  • Backup CLI Configuration
  • Create/Delete Layer 2/3/4 firewall rules using VM Wall
  • Install/Uninstall a vShield manually and from template
  • Configure vShield Manager plug‐in capability
  • Configure VM Flow charts
  • Update vShield Zones
  • Add/Edit/Delete User Accounts
  • Assign rights to a user
  • Add/Delete Application‐Port Pair mapping
  • Execute/Schedule Execution of virtual machine discovery
  • Utilize vShield Zones CLI commands to configure and monitor vShield Zones
  • Analyze traffic using VM Flow to determine root cause of network related issues

Installing vShield Zones

Deployed as an appliance with two components;

  • Setup the vShield Manager appliance
    • Deploy the vShield Manager from OVF
    • Create a port group on the vSwitch which hosts your VM traffic, named vsmgmt and amend the vNIC on the vShield Manager VM to use this network.
    • Power up the VM, login with ‘admin’ and ‘default’, then run ‘setup’ to configure the server.
    • Allocate IP details
    • Upgrade VMtools (you can use the ‘Automatic’ option – being Linux based no reboot is required)
  • Initial install of the vShield Agent
    • Deploy from OVF and then convert to a template. This simply gets the agent ready for deployment.

If you’re wondering whether VMtools make a significant difference to this customised Linux appliance see (the pointless) VMwareKB1011501! You can also find out what’s new in vShield Zones 1.0 Update 1.

Continue reading VCAP-DCA Study notes–7.3 vShield Zones

VCAP-DCA Study notes 7.2– Configure and Maintain the ESX Firewall

A blessedly quick objective this one! Quite why the ESXi Configuration Guide is listed in the blueprint is anyone’s idea as ESXi doesn’t contain a firewall! The blueprint also lists vicfg-firewall which is a typo – they mean esxcfg-firewall, as vicfg-firewall doesn’t exist!

Knowledge

  • Identify vicfg-firewall commands
  • Explain the three firewall security levels
  • Identify ESX firewall architecture with/without vCenter Server

Skills and Abilities

  • Enable/Disable pre‐configured services
  • Configure service behavior automation
  • Open/Close ports in the firewall
  • Create a custom service
  • Set firewall security level

Firewall architecture

The ESX Configuration Guide talks very generally about where to put firewalls to protect traffic. In reality I can’t see much difference in architecture whether you have a vCenter server or not.  These two diagrams are from the ESX Configuration Guide – minimal differences!

The firewall is ESX only (there’s no ESXi firewall as no service console).

imageimage
Firewall security levels

Three firewall security levels (high is default);

  1. High (outbound blocked, limited inbound allowed (902, 443,22,123 and a few other including ICMP).
  2. Medium (outbound allowed, inbound blocked apart from allowed services)
  3. Off

Continue reading VCAP-DCA Study notes 7.2– Configure and Maintain the ESX Firewall

VCAP-DCA Study notes 7.1 – Secure ESX/ESXi hosts

Security is a large topic and one you could spend a lifetime mastering. The blueprint isn’t too helpful in clarifying what level of detail you’re expected to know for this as the ESX/ESXi configuration guides cover issues not in the ‘skills and abilities’ section. More in depth still is the vSphere Hardening Guide. I guess the main thing is to focus on practical issues as the VCAP-DCA is a practical exam – knowing that the VMkernel uses memory hardening is no use in an exam if it can’t be configured or tweaked! Some of this section seems to have been added for the sake of it – how often will an admin need to modify the SSL timeouts? I could only fine one KB article about it!

Knowledge

  • Identify configuration files related to network security
  • Identify virtual switch security characteristics

Skills and Abilities

  • Add/Edit Remove users/groups on an ESX Host
  • Customize SSH settings for increased security
  • Enable/Disable certificate checking
  • Generate ESX Host certificates
  • Enable ESXi lockdown mode
  • Replace default certificate with CA‐signed certificate
  • Configure SSL timeouts
  • Secure ESX Web Proxy
  • Enable strong passwords and configure password policies
  • Identify methods for hardening virtual machines
  • Analyze logs for security‐related messages

Virtual switch security characteristics

vSwitch security (layer2) settings (can be overridden at portgroup level);

  • Promiscuous mode – needed for packet sniffing, vShield Zones (and virtual ESX hosts). Disabled by default.
  • MAC address changes –affects inbound traffic. May need to be enabled if you’re using MS load balancing in Unicast mode, or the iSCSI software initiator with certain storage arrays. Enabled by default.
  • Forged transmits – affects outbound traffic. Enabled by default.

Other network security measures (IPSec, VLANs, PVLANs etc) are dealt with in section 2, Networking.

Host security

Customise SSH settings (ESX only)
  • Edit /etc/ssh/sshd.conf and set ‘PermitRootLogin’ to YES (default is NO). See VMwareKB for a list of other settings you can adjust (including the available ciphers).
  • You can use PKI to authenticate using SSH without being prompted for a password. This is a standard Linux procedure – for step by step instructions see VMwareKB1002866.
  • By default only SSH server is enabled. Configuration -> Security Profile to enable SSHClient, or use ‘esxcfg-firewall –e SSHClient’.
    image

Continue reading VCAP-DCA Study notes 7.1 – Secure ESX/ESXi hosts