<body><script type="text/javascript"> function setAttributeOnload(object, attribute, val) { if(window.addEventListener) { window.addEventListener('load', function(){ object[attribute] = val; }, false); } else { window.attachEvent('onload', function(){ object[attribute] = val; }); } } </script> <div id="navbar-iframe-container"></div> <script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script> <script type="text/javascript"> gapi.load("gapi.iframes:gapi.iframes.style.bubble", function() { if (gapi.iframes && gapi.iframes.getContext) { gapi.iframes.getContext().openChild({ url: 'https://www.blogger.com/navbar.g?targetBlogID\x3d4684235500622716427\x26blogName\x3dCaiwangqin\x27s+blog\x26publishMode\x3dPUBLISH_MODE_HOSTED\x26navbarType\x3dBLUE\x26layoutType\x3dCLASSIC\x26searchRoot\x3dhttp://blog.caiwangqin.com/search\x26blogLocale\x3dzh_CN\x26v\x3d2\x26homepageUrl\x3dhttp://blog.caiwangqin.com/\x26vt\x3d3393395200455623441', where: document.getElementById("navbar-iframe-container"), id: "navbar-iframe" }); } }); </script>

Caiwangqin's blog

Focus on Cloud Service, Smart Hardware, Architecture, Technic and beyond…

Install MogileFS on Debian Sarge (Complete version)

2008年1月7日星期一

先转一篇网上的指引,我的安装过程是在这篇文章下进行的。由于这篇唯一的在Debian平台上安装MogielFS的文档被GFW了,全文转发一下(如果你也需要在Debian上安装MogileFS,建议看完本篇BLOG再进行,后面有补充):



Another How to Install MogileFS - Debian Sarge


This is created from a bunch of sources. You may notice that some of it comes from the other How To. I have tested this many times and it works great!


Doug Kersten


INSTALLING MOGILEFS

If you haven’t already done so add user mogilefsd:


adduser mogilefsd


mkdir /etc/mogilefs


cd /etc/mogilefs


Apt-get install subversion


svn checkout http://code.sixapart.com/svn/mogilefs/trunk/

now you have the code.


apt-get install liblinux-aio-perl


apt-get install libbsd-resource-perl


apt-get install libcompress-zlib-perl


apt-get install libnet-netmask-perl

apt-get install libio-stringy-perl

apt-get install libwww-perl


perl -MCPAN -e shell;


Answer default to all questions until you get to the location information and then choose your country and then your default ftp site.


at the > prompt do this:


install X::Y


Do this with the following:


install Danga::Socket


install DBI


Where X::Y is the name of the missing module. You may have to do this several


times.


exit


Install PerlBal


Mkdir /etc/perlbal


cd /etc/perlbal


svn checkout http://code.sixapart.com/svn/perlbal/trunk


cd /etc/perlbal/trunk


perl Makefile.PL


make

make test


make install


Install MogileFS (mogilefsd and mogstored)


cd /etc/mogilefs/trunk/server


perl Makefile.PL


make


make install


now as root


mkdir /var/mogdata/dev1

As you add more devices add more of these directories


Now for the interesting part.


Install Mogadm (Mogile Utilities):


cd /etc/mogilefs/trunk/utils


perl Makefile.PL


make


make install


CREATE MYSQL DATABASE


You should have mysql installed. I’m going to assume you do.


Using phpadmin create a mog user (mog_user)


create a password for mog user (mogpass)


OR


Grant access to remote user by doing:

$> mysql -u root -prootpassword


$> GRANT usage ON . to mog_user@www.domainname.com (put the correct user/domain(source) here)

$> IDENTIFIED BY ‘mogpassword’;


$> GRANT ALL PRIVILEGES


$> ON . TO mog_uer@www.domainname.com WITH GRANT OPTION;


exit


/etc/init.d/mysql restart


Automatically create a database for the mogilefs:


cd etc/mogilefs/trunk/server


vi mogdbsetup


edit the following:


dbhost => “localhost”,


dbname => “mogilefs”,


dbrootuser => “root”,


dbrootpass => “root_password”,


dbuser => “mog_user”,


dbpass => “mogpass”,


save and run /etc/mogilefs/trunk/server/mogdbsetup


you will be asked a series of questions. Answer yes to them all.


The mogilefs database should be created. Check in phpmyadmin to verify.


Remove the id and password information when you are finished from mogdbsetup.


YOU MUST DO THIS it solves errors about lib path when running mogadm are solved by the following:


Do this - to set up MogileFS.pm go to the /etc/mogilefs/trunk/api/perl directory and do


perl Makefile.PL


make


make test


make install (assuming all goes well).


You may have to cpan install something based on the errors from perl Makefile.PL.


Install Mogilefs utils:


cd /etc/mogilefs/trunk/utils


perl Makefile.PL


make


make test


make install


Configure Mogstored:


On each storage server, create a configuration file at /etc/mogilefs/mogstored.conf with the following:


httplisten=0.0.0.0:7500


mgmtlisten=0.0.0.0:7501


docroot=/var/mogdata


Startup Mogstored:


open a putty session as root and type mogstored. Mogstored should start.


Configure MogileFS:


edit the bold areas of the mogilefsd.conf file:


cp /etc/mogilefs/trunk/server/conf/mogilefsd.conf
/etc/mogilefs/mogilefsd.conf


#daemonize = 1


db_dsn = DBI:mysql:mogilefs:host=127.0.0.1


db_user = mog_user


db_pass = dbpass


conf_port = 7001


listener_jobs = 10


delete_jobs = 1


replicate_jobs = 5


mog_root = var/mogdata


reaper_jobs = 1


launch new putty session ( I use putty: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html


for my ssh sessions to the server.).


login as mogilefsd


at the command prompt run mogilefsd


Use ‘mogadm’ to add each storage server to the database.

The following example would add the host mogilestorage.yourdomain.com as a storage server, assuming that mogilestorage.yourdomain.com had an IP address of 192.168.42.3 (listening on port 7500) and your tracker had an IP address of 192.168.42.1 (listening on port 6001):


# mogadm –lib=/usr/local/share/perl/5.8.4 –trackers=192.168.42.1:6001 host add mogilestorage –ip=192.168.42.3 –port=7500 –status=alive


You can confirm that your host(s) were added with the following command;


# mogadm –lib=/usr/local/share/perl/5.8.4 –trackers=192.168.42.1:6001 host list


You also need to add devices for each storage host. If you don’t provide a device id mogadm is supposed to do it for you… in practice I find it dies with an error, so you will need to manually add a unique device id after the host:

# mogadm –lib=/usr/local/share/perl/5.8.4 –trackers=192.168.42.1:6001 device add mogilestorage 1


Finally, add the correctly-name device (folder) to each storage host. I have been unable to get the tools to handle this well, so I am probably doing something wrong. As a workaround, I used the modadm device list command to see what device names were assigned and then I added the folders to my storage hosts. Run the following command:


# mogadm –lib=/usr/local/share/perl/5.8.4 –trackers=192.168.42.1:6001 device list


It will list each host and the device name followed by its status and storage available. Here is example output:


mogilestorage 1: alive


used(G) free(G) total(G)


dev1: alive 0.892 67.772 68.664


This means “mogilestorage” has a host id of “1″ and it has one device named “dev1″ on it and that device is in the “alive” state (your other statistics will probably be zeros). Using the example output above, you would simply create the directory on mogilestorage.yourdomain.com:

You can also test using mogadm:


mogadm –trackers=127.0.0.1:7001 check


use mogadm to add the host:

mogadm –trackers=127.0.0.1:7001 host add host1

domain:

mogadm –trackers=127.0.0.1:7001 domain add storage

and class:

mogadm –trackers=127.0.0.1:7001 class add storage files.


Go into phpmyadmin, open the hosts table and change the host status to alive and add a device with device id 1, host id 1 and status = alive in the device table.


OR


mogadm –trackers=127.0.0.1:7001 device add device1 1


and mark the device alive


mogadm –trackers=127.0.0.1:7001 device mark host1,cin 1 alive


check your devices:


mogadm –trackers=127.0.0.1:7001 device list


Check it out:


mogadm –trackers=127.0.0.1:7001 check


If everything checks out ok you will see the following:


Checking trackers…


127.0.0.1:7001 … OK


Checking hosts…


1 host1 … OK


Checking devices…


host device size(G) used(G) free(G) use%



1 dev1 138.236 0.143 138.094 0.10%



total: 138.236 0.143 138.094 0.10%


Once everything is working you can daemonize mogilefsd:


edit the mogilefsd.conf file and remove the # in front of daemonize:


vi /etc/mogilefs/mogilefsd.conf


daemonize = 1


db_dsn = DBI:mysql:mogilefs:host=127.0.0.1


db_user = mog_user


db_pass = dbpass


conf_port = 7001


listener_jobs = 10


delete_jobs = 1


replicate_jobs = 5


mog_root = var/mogdata


reaper_jobs = 1


To start mogstored as a daemon type:


mogstored -d


To test and see if it’s mogstored that’s messed up or something along


the way, try telnetting to your mogstored (port 7500 by default) and


doing this:


PUT /dev1/test HTTP/1.0


Content-length: 4


test


See if that creates the ‘test’ file in your /var/mogdata/dev1 directory.


If it works, good, if it doesn’t, you should get back a 404/something with


a more detailed error message.


Now put a file onto mogstored via mogilefsd (tracker):


cd /etc


mogtool –trackers=127.0.0.1:7001 –domain=domainname –class=classname inject motd motd


check /var/mogdata/dev1/0/000/000 for a file named xxxxxxxxxx.fid . If the file exists everything is successful.


Now delete the file:


mogtool –trackers=127.0.0.1:7001 –domain=domainname –class=classname delete motd


check /var/mogdata/dev1/0/000/000 and the file should be gone.


It’s all good!



在以上安装过程需要补充以下几点:



  • 使用cpan安装 String::CRC32 库,这是依赖项(感谢Yupoo的Zola Zhou):


> cpan String::CRC32




  • 在docroot目录下手动建立device目录,如:


# mkdir -p /vag/mogdata/dev1




  • 使用mogadm管理host, device时,先要设置host alive, 才能设置该host下的device为alive:


mogadm –trackers=127.0.0.1:7001 device mark host1,cin 1 alive



标签:

posted by Jesse Cai, 上午2:53

0 Comments:

发表评论

订阅 帖子评论 [Atom]

<< 主页