Justin Waite's Blog
Website on S3 - February 18, 2011 at 12:15 pm
S3 Lab-
For this lab we will demonstrate the ease of hosting static content in S3.
This should not take very long and is very easy, just follow the steps below or watch the video located here http://www.modelmetrics.com/wp-content/uploads/2011/02/DePaul-S3-Lab.swf
1. Open a web browser and go to http://www.freewebsitetemplates.com/ here we will select a free website to get started with, for this demo I will choose the travel template. Select the template site you want to use and download it to you computer.
2. Unzip the download.
3. Log into aws.amazon.com and navigate to the S3 page.
4. In the S3 tab create a bucket. I will use demos3 as the bucket name.
5. Select the bucket you just created
6. Now right click in the Objects and Folders section and choose upload, in the upload manager window highlight the 3 files index.html, style.css and travel.psd then hit open.
7. Now in the Upload Files window at the bottom right click start upload. You should see a progress bar in the transfer’s section and the files should show up in the Objects and Folders screen.
8. Now click the create a folder button, when the new folder pops up name it exactly as the name of the folder that was in the zip file you downloaded, I will use images.
9. Now double click on the images folder to open it.
10. In the images folder click the Upload button.
11. In the upload window click inside the images folder and select all the files in it, then hit the start upload button.
12. You should see files appear in the images bucket.
13. Now navigate back to the bucket you created, mine was demos3
14. Now hight light the index.html object and right click to get to the properties.
15. In the properties you should see 3 tabs- Details, Permissions and Metadata. Lets look at Metadata for the index.html file.
16. The S3 console is pretty good at recognizing the files and we want to make sure it is Key: Content-Type Value:text/html
17. Now click back on details and copy the link and paste it into a web browser, what do you see? It should look like this AccessDeniedAccess Denied80EDC005228C118A4ptOunASFW/ER0Q6QxM52ohdqdmtHuLFrbMlX+skwNS8IcEvGl+BcLxHjdPkdUN6
18. Now lets select all four objects in the bucket and right click and select make public.
19. Now try the link again, what do you see? A website with images? If no, we need to go into the images folder and make all images public.
20. Now remember to delete all the stuff you added or you will be charged.
In this exercise we will show the steps to setting up a AWS EC2 server, connect over ssh and install all the tools and config WordPress. I have attached a video to each step and it might make more sense to read the steps, watch the video and then try them yourself. If you are using windows you will need to follow this post to connec to to AWS.
1. First we will need to log into the AWS console and launch a server. Please log in and launch a server with the following-
a. AMI ID = Ami-84db39ed
b. Instance Type = micro
c. Tag = wp
d. Keypair = wptest
e. Security group ports 22 and 80 from 0.0.0.0/0
Here is a video to watch
2. Connecting to the server we will use terminal. In order to do this we will need to use the PEM we downloaded in step one. We also need to change the permissions of the PEM file with the chmod command. First open terminal and change directories to the location of the PEM we downloaded, mine is in the downloads folder
cd Downloads
Next we need to chmod 400 the PEM file.
chmod 400 wptest.pem
Then last we need to connect to the server
ssh –i wptest.pem root@ec2-12-34-55-23.amazon.com
Here is a video to watch
3. Now we need to install apache, MySQL and some PHP modules. To do this we will use the YUM package manager. The command we will typ in is the following-
Yum install httpd mysql-server phpmyadmin –y
Here is a video to watch
4. More configuration is needed to get WordPress up and running. Now we will download wordpress with the tool wget-
wget http://wordpress.org/latest.zip
Next we need to start MySQL
/etc/init.d/mysqld start
Now we need to run the mysql secure installation
/usr/bin/mysql_secure_installation
Next start apache
/etc/init.d/httpd start
Now we need to modify the configuration file for phpmyadmin.
We open the file with the VI command line editor-
vi /etc/httpd/conf.d/phpMyAdmin.conf
We need to modify the allow from line. In the example we will remove 127.0.0.1 and replace with all, normally you would only allow this from your public IP address using whatismyip.com. After your make this change you need to restart the apache server with the following command –
/etc/init.d/httpd restart
Here is a video to watch
5. Now you need to get wordpress zip file that you downladed and get that in the web directory /var/www/html/. To do this we will use the move command –
mv latest.zip /var/www/html
Now we need to navigate to the lateset.zip
cd /var/www/html
ls -la
Now we need to unzip the file-
Unzip latest.zip
Now move to the uzipped directory
cd wordpress
Now move all the files in the WordPress directory back to /var/www/html
mv * ..
Here is a video to watch
6. Now we will connect to phpMyAdmin to create a DB and a DB user.
First connect to the new EC2 on port 80 and validate that apache is working you should see the apache test page. Second you need to go to phpMyAdmin and enter in the user = root and the password you set up during the mysql secure installation. Third you need to create a WordPress user and a WordPress database.
Here is a video to watch
7. Now we are on to the final step! First you will need to connect to the EC2 server on port 80 to finish the wordpress installation. You will need to use the same user and pass we set up in phpMyAdmin in the WordPress install screen. Next we need to create the wp-config.php using the vi command, all the info we need to use is created for us so we just need to copy and paste the info in to the config file.
vi /var/www/html/wp-config.php
Next click the run install button. Now we are ready to log in to the WordPress admin and start blogging!
Remember you MUST shut down the servers when you are done or you will pay for them!!
So this is an interesting question, many small businesses are finding the cloud extremely useful but how will the enterprises incorporate it? I am sure you have all seen the Microsoft commercials aimed at the consumer “to the cloud!”, and I am sure most of you have used a cloud app before like drop box, Salesforce, Netflix etc but again what is the best way for enterprises to test the water?
Move your development environment to Amazon-
- All developers using the same pay as you go resources. This means that you only pay for the servers that the developer's need at that time, use a script that will boot the boxes at 8AM and stop at 6PM, no more moving 4 year old retired prod boxes to development and wasting money fixing them!
- Developers NOT using resources in a million dollar private data center. Can you imagine if you just built a new data center and things are working great, all new gear humming along. Now look at the utilization across your VMware clusters and SAN, wow 35% of the cluster is getting used for development! So you are stuck in a cycle where you need to prepare for the worst case, say it is 100 million users hitting your app or site, so you build the data center for that and in the first year you are losing 25-35% for development!! Now you’re already planning for next years budget to add more resources, not in the cloud!
- Instantly clone and deploy to every developer!! Taking app and data snapshots have been around for a while and it is not cheap in physical data centers. It is a line item you add at a large cost to the SAN frame and VMware servers so you can hot clone and deploy. Now take a look at the cloud, say developer 1 is just finished coding something and you want to let all 20 developers work off that master code…. easy as a few right clicks on AWS.
- Load testing. Do I really need to say anything here? If you were a developer would you rather go and do your own load testing in the cloud with no impact to anyone or Load test on the 4-year-old hardware in your datacenter and risk slowing down production? Or better yet go and ask your network admin and sys admins when it will be a good time to do some serious load testing; they might chuckle a little bit then say NO.
- Keeping the IP. Lets say you are on the final stretch to getting a golden master of your app or site or whatever it is and you need to bring in some consultants to help out. Say you bring in 10 consultants for 3 weeks. Now they are not employees so they most likely do not get computers owned by your company, they use there own. Now lets also assume that you do not have the time to make a VMware desktop for all of them because it is 10 people for 3 weeks, not worth it. Now 2 weeks in and 2 guys don’t show up anymore and you call the company or agency and they say “yeah sorry about that, this was the first job they worked for us. Guess they did not work out! We will send 2 more out tomorrow morning”. Ok great back up to 10 guys plus your full time staff but wait, the two new guys show up and they sit down and you say, “ OK fellas just start where they guys who left stopped working”. Then realize that they were not checking the code back in to SVN and you do not have the laptops they were working on. Now if you’re having you’re teams developing in the cloud no issues? All the code is up in your companies account, done deal.
Think about it, all developers using pay as you go resources, managing them with out much help from sys admins and have the access to quickly clone any environment and have all developers working on the same stack in your AWS account… perfect.
In this post I will show how to launch a EC2 server, install the tools needed to sync svn repos with a paid svn repository then create a dump file that can be imported to a new SVN server. We were migrating from Codesion, now a CollabNet company to a server that we had built in AWS. Nothing against Codesion they are awesome, we just got to big to continue using it and needed to have more control. I will use codesion again in the future.
Step 1) Launch a EC2 server-
Step 2) Pick your AMI, I like the FC8
.png)
Step 3) Picking the FC8 AMI
Step 4) Choosing the small
Step 5) No advanced instance options
Step 6) Adding a tag
Step 7) Selecting the correct key
Step 
Selecting the correct security group
Step 9) All looks good
Step 10) It's launched
Step 11) Connecting to the new AMI
Step 12) Logging in over SSH
Step 13) Ok now on to the command line stuff. First we are going to install SVN
Administrators-MacBook-Pro-2:PEM justinwaite$ ssh -i MMJWaite.pem root@ec2-184-72-79-138.compute-1.amazonaws.com
The authenticity of host 'ec2-184-72-79-138.compute-1.amazonaws.com (184.72.79.138)' can't be established.
RSA key fingerprint is a2:4c:65:97:2d:c5:3d:56:9d:51:b4:73:e4:0c:05:3f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-184-72-79-138.compute-1.amazonaws.com,184.72.79.138' (RSA) to the list of known hosts.
__| __|_ ) Fedora 8
_| ( / 32-bit
___|\___|___|
Welcome to an EC2 Public Image
Base
–[ see /etc/ec2/release-notes ]–
[root@ip-10-204-213-7 ~]# yum install svn
Loaded plugins: fastestmirror
Determining fastest mirrors
* updates-newkey: kdeforge.unl.edu
* fedora: kdeforge.unl.edu
* updates: kdeforge.unl.edu
updates-newkey | 2.3 kB 00:00
fedora | 2.1 kB 00:00
updates | 2.6 kB 00:00
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
–> Running transaction check
—> Package subversion.i386 0:1.4.4-7 set to be updated
–> Processing Dependency: perl(URI) >= 1.17 for package: subversion
–> Processing Dependency: libneon.so.27 for package: subversion
–> Running transaction check
—> Package perl-URI.noarch 0:1.35-3.1 set to be updated
—> Package neon.i386 0:0.27.2-2 set to be updated
–> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
subversion i386 1.4.4-7 fedora 2.3 M
Installing for dependencies:
neon i386 0.27.2-2 fedora 109 k
perl-URI noarch 1.35-3.1 fedora 116 k
Transaction Summary
================================================================================
Install 3 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 2.5 M
Is this ok [y/N]: y
Downloading Packages:
(1/3): neon-0.27.2-2.i386.rpm | 109 kB 00:00
(2/3): perl-URI-1.35-3.1.noarch.rpm | 116 kB 00:00
(3/3): subversion-1.4.4-7.i386.rpm | 2.3 MB 00:00
——————————————————————————–
Total 3.3 MB/s | 2.5 MB 00:00
============================== Entering rpm code ===============================
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : neon 1/3
Installing : perl-URI 2/3
Installing : subversion 3/3
=============================== Leaving rpm code ===============================
Installed:
subversion.i386 0:1.4.4-7
Dependency Installed:
neon.i386 0:0.27.2-2 perl-URI.noarch 0:1.35-3.1
Complete!
14. Next we are going to create the repo-
[root@ip-10-204-213-7 ~]# svnadmin create /tmp/mm_framework
15. Now we need to create the pre-revprop-change in the hooks directory –
[root@ip-10-204-213-7 ~]# touch /tmp/mm_framework/hooks/pre-revprop-change
16. Next change permissions –
[root@ip-10-204-213-7 ~]# chmod +x /tmp/mm_framework/hooks/pre-revprop-change
17. Next we need to run the initial sync against the SVN server -
[root@ip-10-204-213-7 ~]# svnsync init file:///tmp/mm_framework/ https://mmetrics.svn.cvsdude.com/MM_2GO_frameworkFieldService
Authentication realm: <https://mmetrics.svn.cvsdude.com:443> mmetrics login
Username: mmetrics
Password for 'mmetrics': XXXXXX
Copied properties for revision 0.
18. Now we are going to sync the EC2 svn with the hosted SVN-
[root@ip-10-204-213-7 ~]# svnsync sync file:///tmp/mm_framework/
Authentication realm: <https://mmetrics.svn.cvsdude.com:443> mmetrics login
Password for 'root':
Authentication realm: <https://mmetrics.svn.cvsdude.com:443> mmetrics login
Username: mmetrics
Password for 'mmetrics':
Committed revision 1.
Copied properties for revision 1.
Committed revision 2.
Copied properties for revision 2.
Committed revision 3.
Copied properties for revision 3.
Committed revision 4.
Copied properties for revision 4.
Committed revision 5.
Copied properties for revision 5.
Committed revision 6.
Copied properties for revision 6.
Committed revision 7.
Copied properties for revision 7.
Committed revision 8.
Copied properties for revision 8.
Committed revision 9.
Copied properties for revision 9.
Committed revision 10.
Copied properties for revision 10.
Committed revision 11.
Copied properties for revision 11.
Committed revision 12.
Copied properties for revision 12.
Committed revision 13.
Copied properties for revision 13.
Committed revision 14.
Copied properties for revision 14.
Committed revision 15.
Copied properties for revision 15.
Committed revision 16.
Copied properties for revision 16.
Committed revision 17.
Copied properties for revision 17.
Committed revision 18.
Copied properties for revision 18.
Committed revision 19.
Copied properties for revision 19.
Committed revision 20.
Copied properties for revision 20.
19. Now we need to export the data using the SVN dump command-
[root@ip-10-204-213-7 ~]# svnadmin dump file:///tmp/mm_framework/ > ~/mmdump.dmp
svnadmin: 'file:///tmp/mm_framework' is an URL when it should be a path
[root@ip-10-204-213-7 ~]# svnadmin dump ///tmp/mm_framework/ > ~/mmdump.dmp
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
* Dumped revision 3.
* Dumped revision 4.
* Dumped revision 5.
* Dumped revision 6.
* Dumped revision 7.
* Dumped revision 8.
* Dumped revision 9.
* Dumped revision 10.
* Dumped revision 11.
* Dumped revision 12.
* Dumped revision 13.
* Dumped revision 14.
* Dumped revision 15.
* Dumped revision 16.
* Dumped revision 17.
* Dumped revision 18.
* Dumped revision 19.
* Dumped revision 20.
20. Last we will verify the file –
[root@ip-10-204-213-7 ~]# ls -la ~
total 328
drwxr-x— 3 root root 4096 Jan 12 18:06 .
drwxr-xr-x 22 root root 4096 Jan 12 16:57 ..
-rw-r–r– 1 root root 24 Feb 22 2008 .bash_logout
-rw-r–r– 1 root root 327 Feb 22 2008 .bash_profile
-rw-r–r– 1 root root 184 Dec 11 2007 .bashrc
drwx—— 2 root root 4096 Jan 12 16:57 .ssh
-rw-r–r– 1 root root 303870 Jan 12 18:06 mmdump.dmp
21. Well we have an snv dump of the repo on the hosted server and we can import to a new server.
TeamCity Distributed Build Management and Continuous Integration Server
So a few months ago we installed Teamcity on an ec2 server to mess around with. It was really just for us to see how well it works and we loved it so much we have not stopped using it! So I thought I would put up the install docs.
Teamcity install
Getting the Nod-
Dreamforce/Cloudstock
I was extremely lucky to attend Dreamforce and Cloudstock in December and was excited to see all the multi-cloud love. In the keynote given at Dreamforce, Salesforce announced a few new multi cloud offerings; database.com, VMforce, Heroku, and they gave the nod to Amazon Web Services with a live demo. In the demo they had an ec2 box running PHP, talking to database.com. It was also obvious at Cloud stock that multi-cloud was all over, Engine Yard was in the house, Jeff Barr from Amazon was in the house, Cloud Kick in the house, Google in the house, mogoDB, Twilio, VMware and so many more, check it out – http://www.cloudstockevent.com/sessions.
This was really interesting to see that the leader in cloud/SAAS letting the world see who it’s endorsing.
Oracle
Oracle gets it. Once again Larry and team give AWS the nod release the following applications as AMIs for use with EC2:
- Oracle PeopleSoft CRM 9.1 PeopleTools
- Oracle PeopleSoft CRM 9.1 Database
- Oracle PeopleSoft ELM 9.1 PeopleTools
- Oracle PeopleSoft ELM 9.1 Database
- Oracle PeopleSoft FSCM 9.1 PeopleTools
- Oracle PeopleSoft FSCM 9.1 Database
- Oracle PeopleSoft PS 9.1 PeopleTools
- Oracle PeopleSoft PS 9.1 Database
- Oracle E-Business Suite 12.1.3 App Tier
- Oracle-E-Business-Suite-12.1.3-DB
- JD Edwards Enterprise One – ORCLVMDB
- JD Edwards Enterprise One – ORCLVMHTML
- JD Edwards Enterprise One – ORCLVMENT
Believe it or not you can use your existing Oracle licenses at no additional license cost or you can acquire new licenses from Oracle.
So in the end of 2010 we see Salesforce and Oracle give the nod to AWS. All I can say is buckle up because 2011 is going to be moving, who is next?
So I decided i would start putting up some white papers or at least screen captures of basic installs that I do often. So with out adding too much hype here is the first one, Jboss, MySQL and httpd on Centos.
This is a pretty straight forward install and does not touch on the AWS setting etc, more just the OS and install of the apps.
jboss-doc
-Justin
Route53 - January 3, 2011 at 9:19 am
Route 53-
So this week I spent some time setting up route53 on a domain we have for testing and I must say it was pretty simple but was harder then it should be. I started out reading the documentation and downloading the tools to get it started and was surprised how difficult AWS was making route 53. So most of my DNS experience is with Microsoft, godaddy, Mac OS x, Bind and some free DNS tools, now all of those I mentioned had some sort of GUI based tools so I was thinking maybe I was just spoiled? Well I started off with the getting started guide and went to step 1- creating a hosted zone. I was instantly surprised to see that I needed to download a dnscurl.pl script, then create an xml file to process the hosted zone config. So I worked over the docs and finally got the xml file correct and the aws access keys correct then after a few attempts got the script to run. So that was not that hard, needed some prep but was able to get it to go from my Mac, now on to step 2.
Step two creating a new dns record-
So I am back to creating a second xml file that holds all the info in it for me to set up some A records, TTL all that good dns stuff. Step two took about twice as long as step one but I was able to work over the documentation and troubleshoot some issues with xml and get it to go.
Step three confirm you request is complete-
So step three was painless I am finally getting the hang of how this works and was able to get it to display the correct info on the second try.
Step four update the registrar’s NS records-
This step I felt in control again was able to get it on the second try and then waited for the info to propagate so I could see the correct DNS info on a dig and nslookup.
That’s it all done!
Now on to why- So I am not sure if I am a early adapter or what but I was happy to see that this can be done from the command line however I would say this took me about 2 hours longer then putting the info in godaddy! I am sure that AWS will implement this in the aws console pretty quick since it is really just some fields in a web site, but it made me feel a little overworked! I guess this is the way that we will still see new products from AWS, I remember setting up ELB’s and RDS stuff from the command line so this is par for the course.
All in all the process was fairly simple and the second and third time I do this I will probably get it in 30 – 45 minutes but until AWS get it in the console it will always take longer then any of the legacy gui tools that I have used for years.
I think that as soon as Route53 comes out of beta we will see all the enhancements that we need and Route53 will be a great way to manage DNS.
Boom, just like that AWS changes the game again. Last night they announced VM-Import. As of now they can only import 32 and 64 bit windows 2008, however they are working on geting the "nixs" working.
Can you imagine what this means for diaster recovery? Or what this will do to let enterprises test the waters? This is a big step for supporting the hybrid-cloud approach, making the public cloud more translucent to enterprises leveraging the largest private cloud VMware.
More to come, was so excited i had to get this out..
Here is a link to Jeff Barr's post where he gets into more detail, but what a way to kick off my blog here at MM
-Jusitn Waite
Blog
Contact Us