Original paper: http://static.googleusercontent.com/media/research.google.com/en//archive/gfs-sosp2003.pdf Cons and pros on Quora: https://www.quora.com/Reviews-of-Google-File-System Paper discussion: http://pages.cs.wisc.edu/~swift/classes/cs736-fa08/blog/2008/11/the_google_file_system.html A blog discussion: http://kaushiki-gfs.blogspot.com/ A ppt discussion: http://www.slideshare.net/tutchiio/gfs-google-file-system A QA series: http://pages.cs.wisc.edu/~thanhdo/qual-notes/fs/fs4-gfs.txt GFS wiki: http://google-file-system.wikispaces.asu.edu/ 2.3 GFS uses one-master-mutiple-chunkservers mode. Client will query the master about the metadata of files, which reside in the master’s memory hence query is fast. The real data operation takes place at chunkservers. …
Category Archives: Network Technology
Compile Hadoop 2.7 on Eclipse on Ubuntu 15.04
Install libtool: sudo apt-get install libtool Install protoc 2.5: http://stackoverflow.com/questions/29797763/how-do-i-install-protobuf-2-5-on-arch-linux-for-compiling-hadoop-2-6-0-using-mav Git clone hadoop-common and compile using Maven then import into Eclipse: https://wiki.apache.org/hadoop/EclipseEnvironment Make sure Eclipse uses officially supported JRE: https://wiki.apache.org/hadoop/HadoopJavaVersions, http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Ftasks%2Ftask-add_new_jre.htm You may still find some class cannot be resolved, or some import package can’t be resolved. This is due to that Maven generates some …
Continue reading “Compile Hadoop 2.7 on Eclipse on Ubuntu 15.04”
Very important to set verbose output for Ubuntu User
I often mess up with Ubuntu System so much that I can’t manage to boot the system. I always end up with being stuck at the splash screen with no clue what is going on in the background. So, I suggest that for every ubuntu user you should turn on verbose output during the boot. …
Continue reading “Very important to set verbose output for Ubuntu User”
How to remote access mysql via ssh
You need to let mysql listening on the whole Internet (or the IP address you want to restrict to). By default mysql only allows inbound traffic from localhost. To configure that, first you need to locate the configuration file that mysql takes mysql –help –verbose | more # You will find a paragraph like: Default …
How to run a PPTP VPN server?
Setting up a VPN using PPTP protocal can be vulnerable for security. Yet it is an easy and quick way to set up a virtual private network (VPN). The procedure to setup up a PPTP VPN server can be found here: https://help.ubuntu.com/community/PPTPServer The way to connect to a PPTP server on a client machine can …
Change mongodb data path to portable hard drive
Background I have a 1TB portable usb hard drive which I want to host mongodb data. The OS is ubuntu 15.04 Procedure stop your mongodb service sudo service mongod stop edit `/etc/fstab` so that the computer automatically mounts your portable drive under “mongodb” user everytime the OS boots up: UUID=018483db-1b1c-4dd2-a373-58a6aa5a10be / ext4 errors=remount-ro …
Continue reading “Change mongodb data path to portable hard drive”
Create mongodb replica sets on different machines
Important prerequisites: Confirm connectivity between machines you want to set as replica set Confirm the same version of Mongodb is installed on each machine. I tried one with mongo 2.6 and another with mongo 3.0 and totally failed. Then I installed mongo 3.0 on every machine. Don’t set authentication for all the machines before you …
Continue reading “Create mongodb replica sets on different machines”
Microsoft Word Tips
Tips again. This time is for Microsoft Word. 1. Ctrl+Shift+8 Show non-printing characters. You can also toggle it on manually in the menu. 2. Pay attention to the extension symbol in ribbon. It contains more options. Or you can see paragraph settings in Page Layout in ribbon: 3. Sometimes, you will find …
Install Latex distribution, editor and packages on Ubuntu
1. Install TexLive. This is the latex environment. https://www.tug.org/texlive/quickinstall.html We recommend to use .install-tl script to install TexLive. TexLive is usually installed at “/usr/local/texlive/year/bin/x86_64-linux/….” Although you can install vanilla TexLive via sudo apt-get install texlive, we often observe it misses advanced accessories such as fonts. 2. Install Texmaker http://www.xm1math.net/texmaker/ *** Note: On Ubuntu, Texmaker can …
Continue reading “Install Latex distribution, editor and packages on Ubuntu”
Unclean uninstallation of fcitx causes login loop on Ubuntu
Whether you realize or not, fcitx, a popular input method, if not properly uninstalled, can cause the notorious login loop issue on Ubuntu (i.e., every time you type your password and enter, you get asked to type your password again on the same login window). Here is the post talking about how to fully uninstall …
Continue reading “Unclean uninstallation of fcitx causes login loop on Ubuntu”