Friday, 22 November 2013

Centos 6/RHEL update system with Yum

Updating your system and installing packages is fairly straightforward using the Yum package manager a tool developed by Seth Vidal, who sadly died in a hit and run accident in Durham, North Carolina. 

You will be required to access the server with full root privileges in order to do this and some updates may require a full system reboot.

To begin with, see whether there are any updates for your installed packages.

As root enter in to the terminal

# yum check-update


If no updates are available, then the update process will end and no further work will need to be done. However, if updates are available, Yum will now return a list of all package updates from the repositories known to your system.

Yum update Centos 6
To complete the updateprocess type

# yum -y update

No further work is required so you may resume typical operations. However, if a new kernel has been installed or an important security update has taken place, it may be necessary to reboot the system in order for the new changes to take effect. 

To do this, type 

# reboot  

Yum is the default package management system for CentOS and part of its role is to automatically calculate what packages may require updating, what dependencies are required, and to manage the entire process of updating your system. 

Yum makes the process of updating your system very simple and is easy to use. Additionally, you can use yum-priorities & extra repositories to enhance the whole yum experience in Centos 6.

Thursday, 21 November 2013

Centos 6/RHEL cleaning Yum cache

It is recommended to regularly clean the cache of YUM to ensure that the working cache remains current.

Yum will create a cache that consists of metadata and packages. These files are very useful, but over time they will accumulate in size to such an extent that you will probably find that Yum is acting erratically.

The frequency of this happening can vary from system to system but it generally implies that the Yum cache system requires your immediate attention.

We look at a quick solution that will serve to assist you in cleaning the cache and restoring Yum to its original working state.


First, clean any cached package information, log in as root

# yum clean packages

Now remove any cached XML-based metadata

# yum clean metadata


Remove any cached database files

# yum clean dbcache


Following this you will want to clean all the files to confirm the preceding instructions and to ensure that unnecessary disk space is not used

# yum clean all


Rebuild the Yum cache by typing

# yum makecache


Yum is a very powerful tool that is known for its ability to resolve package dependencies and automate the process of package management, but as with all things, there are times when even the best utilities can get confused and may report errors or behave erratically.

Fixing this issue is relatively simple and the approach outlined will help keep your package manager in a healthy running state for the life time of your operating system.

During its typical operation Yum will create a cache of metadata and packages found in /var/cache/yum. These files are essential, but as they grow in size this cache will slow down the overall use of this utility and may even cause some issues.


To address this situation we started by using the following command to clean the current package-based cache.


We then followed this by cleaning the metadata cache, which would remove any excess XML-based files.


Yum uses SQLite as a part of its normal operation, so the next step was to remove any remaining database files.

The penultimate step was to clean all files associated with enabled repositories in order to reclaim any unused disk space.


Finally, we restored Yum to its normal working state by rebuilding the cache.


So you run through the steps above in order to keep Yum in an optimal working state.


Tuesday, 19 November 2013

Centos 6/RHEL set file permissions

File permissions are an essential component of a Linux system and are used to implement security measures on a Server or Desktop system.

For any file or folder there are three types of access, or permission.

read (from the file)

write (to the file)

execute (run or access the file or folder).

When you do a long file listing (ls –l) of any folder, you will see listed the file permissions for each file. 

As you can see from the sample listing, ten ‘characters’ are used to represent the file permission of each file. For all but the first position, there are five possible characters


r, representing the read permission

w, representing the write permission

x, representing the execute permission

s, which is only found in the execute field (read on) and is used to set the ‘user id’, which gives them access to system resources

-, meaning no permission granted.

Each of the ten characters and its left-to-right sequence has a meaning see below.
The ten character permissions are divided into four groups: 

File type

a.

d represents a directory (or folder)

b.

- represents an ordinary file

c.

s represents a socket used in network communication

d.

p represents a pipe used in processor threads

e.

c and b represent characters and blocks for device-based data buffers. It is rare that you see any of these, so while it is not critical that you commit these to your long-term memory, it is important that you are able to ‘look these up’ in the future.
User permissions define how the owner (anton in the example above) may access, manage or use the file. 

Other user permissions define how the rest of the world may access, manage or use the file.

So in the image above

As an example, Table 11 lists permissions and the resulting access.

Linux file permissions examples

Permission
Access
-r--r-----
The user and the given group only have read access to the file
drwx------
The directory is available (to read, write or enter) for the user only
----rwxr--
Members of a group have full access to the file; other users have read access
----r-----
Only group members can read this file


To change permissions for any file, directory or collection of files (or folders) in Linux, you must use the change mode (chmod) command.

The command has the format

chmod who=/-/+permissions filename

where ‘who’ is …

u, the user who owns the file

g, the group the file is a member of

o, all other users

a, all of the above (a is an abbreviation for ugo).

With the symbols =, - and + you can equate (also known as assign), remove or add permissions. Permissions are r, w and x, a combination of all three, or none.

The chmod command can also assign the file permissions using octal values (octal is a number system from 0 to 7).

 Octal values for permissions

Permissionrwx

binary ‘values’421octal valueresulting file permission

0000---

0011--x

0102-w-

0113-wx

1004r--

1015r-x

1106rw-

1117rwx

If you wanted to set the permissions of a file called 
testfile.sh to rwxr--r--, you would use

chmod 744 testfile.sh

which is the same as the following 

chmod a=r,u=wx testfile.sh

The user executing the chmod command must be the owner of the file or root. 

If the user is root, then they can change any file even if the permission is set to 000 or ---------.

One especially dangerous option to chmod is –R which will recursively change all the permissions for a given folder and all files and folders below it.

So avoid that one unless absolutely necessary.

Wednesday, 13 November 2013

Centos 6/RHEL install Hadoop Multi Node Server

Installation of Hadoop on your Centos 6/RHEL box is now a lot simpler since rpm versions have been made available but you nonetheless need to have installed the JDK prior to doing so.

Change the JAVA_HOME path to /usr/java/default and you can install Hadoop via yum from Epel repo.

$ sudo yum -y install hadoop

If you have any problems with yum you can also use the Apache mirror service, download your preference and install it with

$ sudo rpm -Uvh <rpm_package_name>

Once installed as a package, set  it all up.

Generate hadoop configuration on all nodes

$ /usr/sbin/hadoop-setup-conf.sh

--jobtracker-url=${jobtracker}:9001 \

--conf-dir=/etc/hadoop \

--hdfs-dir=/var/lib/hadoop/hdfs \
   
--namenode-dir=/var/lib/hadoop/hdfs/namenode \
   
--mapred-dir=/var/lib/hadoop/mapred \
   
--datanode-dir=/var/lib/hadoop/hdfs/data \

--log-dir=/var/log/hadoop \

--auto

Where ${namenode} and ${jobtracker} should be replaced with hostname of namenode and jobtracker.

Format namenode and setup default HDFS layout.

$ /usr/sbin/hadoop-setup-hdfs.sh

Start all data nodes after stopping first.


$ /etc/init.d/hadoop-datanode start

Start job tracker node.
   
$ /etc/init.d/hadoop-jobtracker start

Start task tracker nodes
   
$ /etc/init.d/hadoop-tasktracker start

Create a user account on HDFS for yourself.
   
$ /usr/sbin/hadoop-create-user.sh -u $USER

Set up Hadoop Environment
   
$ vi ~/.bash_profile

In INSERT mode set path for JAVA_HOME

Export JAVA_HOME

Save file by clicking esc:wq

Run the .bash_profile
   
$ source ~/.bash_profile

Set JAVA_HOME path in Hadoop Environment file
   
$ sudo vi /etc/hadoop/hadoop-env.sh

Configure Hadoop

Use the following:

$ sudo vi /etc/hadoop/core-site.xml:

<configuration>

<property>
   
<name>fs.default.name</name>

<value>hdfs://localhost:9000</value>

</property>
   
</configuration>

conf/hdfs-site.xml:

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

</configuration>

conf/mapred-site.xml:

<configuration>

<property>

<name>mapred.job.tracker</name>

<value>localhost:9001</value>

 </property>

</configuration>

Hadoop Commands

$ hadoop

$ hadoop namenode –format (Format the namenode, If ask to
answer ‘Y’)
   
$ hadoop namenode (Start the namenode)
   
$ find / -name start-dfs.sh (find the file in directory)
   
$ cd usr/sbin (Go to respective directory directly)
   
$ start-dfs.sh
   
$ start-mapred.sh

$ hadoop fs –ls / (Shows the HDFS root folder)
   
$ hadooop fs –put input/file01 /input/file01 (Copy local input/file01 to

HDFS root /input/file01)


Saturday, 2 November 2013

Centos 6/RHEL install Lamp using Centos Live CD

There can be occasions when you need to use the Centos Live CD if you are away from your usual PC. You may carry about a laptop or you may simply need to use a machine which has neither Linux nor a web server installed on it.

You can set up a test Lamp server using the Centos 6 Live CD in order to try things out. This is surprisingly easy to accomplish as the basic repos which ship with the Live CD carry all the tools you need.

Obviously you will first need to boot the laptop or machine from the Centos Live CD and log in, so this presumes you have already burned it to a disk and done that.

Once you are up and running, first you will need Httpd web server, which is the newer name for the Apache, so install it with the following command.

$ sudo yum -y install httpd

Start it up with

$ sudo service httpd start

We do not set to start on reboot for the obvious reason that the Live CD is not persistant, so this is a one use scenario and as of writing you will need to keep the machine on or in sleep mode and not switch it off before you have finished what you are doing.
Now install mysql with a similar command

$ sudo yum -y install mysql-server

and again start it with

$ sudo service mysqld start

Run the secure configuration tool

$ /usr/bin/mysql_secure_installation

And enter your preferences at the prompt

At this point you can check the installation with firefox if it is installed.

/usr/bin/firefox http://localhost

And you should see a screen such as the one below.


To use php you need to make a couple of small adjustments in the configuration files.

$ sudo vi /etc/httpd/conf.d/welcome.conf

Scroll down to comment out or remove the block as indicated,

$ sudo vi /etc/httpd/conf/httpd.conf

Scroll down to 'DirectoryIndex' and add the line 'index.php'

Now add the final php component

$ sudo yum -y install php php-mysql

Which will give the basics for php use

You can create an info file in the web directory

$ sudo vi /var/www/html/info.php

and add the line

<?php
phpinfo();
?>

Close and save the file then restart the httpd server

$ sudo service httpd restart

Then again browse to http://localhost


So, if you are on the move but need quick access to a webserver, whatever the machine is loaded up with, you can still use Centos with a basic (or more advanced) Lamp stack in minutes.

This is especially useful for anyone using CMS such as Wordpress, Joomla or similar which rely upon the PHP component of the stack