Automated Deployment with EC2 and Bitbucket

First, I am going to split this into several parts in order to be able to handle the wordpress editing process.

1. I am going to describe how I bring up an ec2 instance with boto3
2. I describe the oauth process to the bitbuckt REST API and the transfer of a deploy-key
3. Bring it all together and wrap it up

So, to start lets create an EC2 instance:


import boto3

def start_ec2_app():

c = get_client()
res = get_resource()

app_sg = create_app_sg(c, 'app')

keypair = c.create_key_pair(KeyName='app_key')
with open('keys/'+keypair['KeyName']+'.pem','w+') as keyfile:
keyfile.write(keypair['KeyMaterial'])

inst = res.create_instances(
ImageId = AMI,
KeyName = 'app_key',
InstanceType = INSTANCE_TYPE,
SecurityGroups = ['app'],
MinCount = 1,
MaxCount = 1
)

# let's wait for the instance
runningWaiter = c.get_waiter("instance_running")
runningWaiter.wait(InstanceIds = [inst[0].id])

instances = res.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
i = 0
for instance in instances:
tags = [{"Key" : 'instanceName', "Value" : 'app_%s' %i}]
print(instance.public_ip_address+' '+instance.public_dns_name)
c.create_tags(
Resources = [instance.id],
Tags = tags)

allow_ssh(c, 'app')

def allow_ssh(c, name):
sg = c.describe_security_groups(Filters=[{'Name': 'group-name', 'Values': [name]}])
group = sg['SecurityGroups'][0].get('GroupId')
c.authorize_security_group_ingress(
IpProtocol = "tcp",
CidrIp = "0.0.0.0/0",
FromPort = 22,
ToPort = 22,
GroupId = group)

def get_client():
return boto3.client(
'ec2',
region_name = AWS_REGION,
aws_access_key_id = AWS_ACCESS_KEY,
aws_secret_access_key = AWS_SECRET_KEY,
)

def get_resource():
return boto3.resource(
'ec2',
region_name = AWS_REGION,
aws_access_key_id = AWS_ACCESS_KEY,
aws_secret_access_key = AWS_SECRET_KEY,
)

def create_app_sg(c, name):
sg = c.describe_security_groups(Filters=[{'Name': 'group-name', 'Values': [name]}])
if not sg['SecurityGroups']:
c.create_security_group(
GroupName = name,
Description = '%s Security Group' %name)

start_ec2_app()

Okay, okay… let’s go slowly:
a) we need a client to work (or at least I prefer a client, you could use a resource with some fiddling), so we create one
b) this client now creates a security group that gets the fabulous name ‘app’
c) we run an instance (a single one in this case)
d) we get the id of the instance so we can
e) wait until the instance is running
f) then we can retrieve the public ip and the dns-name
g) lastly we create a pair of keys and save the private key

So, this is done. Off to bitbucket and oauth next…

Advertisements

Perl vs. Python RegEx Shootout

I am constantly told that Perl has much better regex performance than python. When I ask people how they know they answer with “everybody knows that” or “because it’s native” or I am shown some obscure benchmarks whcih seem to test anything but regex performance (hardcoded regex vs interpolated etc.). I wanted to know, and I wanted to fiddle around with performance analysis since I am dealing with Big-O lately. So, without putting an end to the discussion and more as a base for discussions with colleagues and friends here is what I did:

1. I took a large text (Moby Dick at archive.org

2. I wrote a very small programs in perl and python

3. I read in the whole file and measured the time (to be able to see whether one program takes longer to read or not)

4. I ran the code with regex

5. I changed the regex and ran it again

6. I measured with linux’s “time”

I am however not interested in absolute performance (which is machine dependent) but relative.
Version were
perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level
and
Python 2.7.6 (default, Jan 17 2014, 15:43:59) [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin

The first two scripts were these

import re;

count = 0
with open('mobydick.txt','r') as f:
data = f.read();


#!/usr/sbin/perl -w
use utf8;
use strict;
use warnings;

my $string;

open FILE, "<", "mobydick.txt";
$string = join("", );
close FILE;

Ran them both and got
python py_regex.py 0,02s user 0,02s system 53% cpu 0,069 total
perl pl_regex.pl 0,01s user 0,02s system 70% cpu 0,047 total

Pretty close. So, I don’t have to concern myself with reading speed in the next measurements.

Then I changed the code to include some regexes. I just counted how many times the word “Pequod” was used.

import re;

count = 0
with open('mobydick.txt','r') as f:
data = f.read();

m = re.findall('(Pequod)', data);

for find in m:
print find
count+=1

print "%d" %count

#!/usr/sbin/perl -w
use utf8;
use strict;
use warnings;

my $count = 0;
my $string;

open FILE, "<", "mobydick.txt";
$string = join("", );
close FILE;

my @m = $string =~ /(Pequod)/g;

foreach(@m){
print "$_\n";
$count++;
}

print $count."\n";

Ran them again and got:

Pequod
[...]
Pequod
66
python py_regex.py 0,02s user 0,01s system 89% cpu 0,033 total

And

Pequod
[...]
Pequod
66
perl pl_regex.pl 0,01s user 0,01s system 89% cpu 0,021 total

Okay, that was a little surprising since in the discussions I had before “outperforms” was a term used quite often.
Maybe it was just that the regex was simply not complex enough or something…

Change the regex and keep everything else.

m = re.findall('(.*Pequod:*)\s', data);

my @m = $string =~ /(.*Pequod.*)\s/g;

And run it again

the Pequod. Devil-Dam, I do not know the origin of ;
[...]
SLOWLY wading through the meadows of brit, the Pequod
66
python py_regex.py 0,07s user 0,01s system 95% cpu 0,082 total

Not too bad an increase.

the Pequod. Devil-Dam, I do not know the origin of ;
[...]
SLOWLY wading through the meadows of brit, the Pequod
66
perl pl_regex.pl 18,16s user 0,09s system 99% cpu 18,347 total

GOODNESS ME!!

I still don’t know what happened, but I will ask around…