hadooop ambari administration: Important linux commands

Vi/Vim commands:

?/ (search from bottom to top)
/(search from top to bottom)
N(press “n ” for next occurrence of search)
Dd((Of the current cursor positiondelete entire line)
Dd 5((Of the current cursor position delete five lines)
Yy(Of the current cursor position copy the entire line)
Yy 5(Of the current cursor position copy 5 entire line)
P((Of the current cursor position paste the entire line copied)
yw(Of the current cursor position copy the current word)
p((Of the current cursor position paste the current word copied)
dw(Of the current cursor position delete the current word copied)
o(new line below the current cursor)
shift + o(Of the current cursor position newline above empty)
gg(Of the current cursor position go to top of file starting at first position of record)
shift gg(Of the current cursor position go to bottom of file to first position of record )
cntrl + o ( to beginning of file)
:%s/findwordtoreplace/replacetheexistingword(find and replace existing word with replace word)
:wq(write and quit)
:q!(quit without saving)

:set nu (sets number for the lines in vi editor)

MBR--Master boot record

1. GRUB-grand unified boot loader

2. Shell:> init 6 (restart will restart the system in runlevel 6)

3. configuration files for initialization /etc/initab

4. service are in /etc/init.d directory

5. /etc/rc.d ( for each runlevel there will be a separate directory in which necessary services are started as If defined )

6. Shell:>last -10(last 10 logins in the system)

7. Shell:>lastb (last bad login of users)

8. Shell:>cd /etc/audit(audit directory for the audit daemon)

9. auditd.conf(This file controls the configuration of the audit daemon)

10. audit.rules

11. shell:>ausearch --session 1 (the session details with session id given as “1”)

12. shell:>aureport(audit report)

13. shell:>id (gives the current user uid, gid and so on)

14. shell:>id kumar(gives the user uid, gid and so on for user kumar)

15. shell:> cut -f 1,3,4 -d : samplepasswd(cut and display the required fields in the file)

16. shell:> cut -c 1-3 samplepasswd(cut the characters and display)

17. shell:> grep 'bin/bash' ./samplepasswd(Get the output from the file which has the lines ‘bin/bash’)

18. shell:> grep 'bin/bash' ./samplepasswd | cut -d ':' -f 1-6 samplepasswd

19. shell:> awk [ -F fs ] [ -v var=value ] [ 'prog' | -f progfile ] [ file ... ]

20. -F = field separator for regular expression fs

21. -v = Assaigns the value to the variable var before executing the awk program

22. Prog =An awk program

23. -f= awk program in a file to be executed

24. File = the file name to be operated on.

25. Shell:> awk ‘print $0,$3,$4’ Deptssaleamounts.txt

26. Shell:>wc Deptssaleamounts.txt(give the printout of line, words, bytes count for each file)

27. Shell:>netstat –r (routing tables)

28. Shell:>netstat –u (udp connections), netstat –t(tcp connections), netstat –l(listening ports),netstat –p (program name),netstat –g (IPv6,IPv4 group memberships),netstat –I (kernel interface tables),netstat –a

29. Shell:>passwd(change password for current user)

30. Shell:>cat /home/kumar/.bash_history | less

31. Shell:> aulastlog(gives the last login information , need to be a super user to execute this command)

32. Shell:>grep -options 'regex' <filename>

33. Shell:>watch –interval 120 datetime( watch and prints output on console for a program to for the specified interval)

34. Shell:>tail –n 5 passwd(prints the output of last 5 lines on the console)

35. Shell:>adduser kumar (needs to be super user to execute this command), use passwd to set a password for the user

36. Shell:>Lastlog (last logins on the system)

37. Shell:>Usermod [options] <login>

Options are –d , - - home (to change home directory)

38. Usermod -e (set expire date)

39. Usermod –g users kumar (set a group )

40. Usermod –a -G admin Kumar (set Kumar to supplementary, or secondary groups without disturbing primary group)

41. Shell:> Tty (prints the file name of tty connected to standard input)

42. Shell:>Ps --help

43. Shell:>ps –ef(list all process “–e” running on a system with full format “ –f” listing.

44. Shell:>Ps aux(gnu style )(“-a” list w/ tty except session leaders ,” –u” by effective user id,”-x” processes w/o controlling ttys)

45. Shell:>Ps T (all process running on this terminal)

46. Shell:>Ps –u root, ps –u(list all the processes by effective user id)

47. Shell:>ps aux | grep mysqld

48. Shell:>pgrep sshd

49. Shell:>pidof bash (find process id of running program)

50. Shell:>Pidof sshd

//pid -fu samuel

51. Shell:>kill -9 1540(sshd pid is 1540)

52. Shell:>free –ml (show high and low”-l” memory statistics in “-m” mb)

53. Shell:>fuser –muvf /home/kumar/exportfromhive (identify process using files or sockets)

54. Shell:>top (interactive output of linux tasks /processes running with cpu and memory usage

55. Shell:>top -u samuel

56. Shell:>service httpd start(start httpd service which is webserver)

57. Shell:>ftp 192.168.50.24 (ftp to address)

58. Shell:> sftp 192.168.50.2 (sftp to address)

59. ftp>mput *.txt (put file from current directory of type *.txt onto ftp site)

60. ftp>mget *.txt

61. Shell:>sort -r filea.txt(reverse the result in descending order)

62. tar -cvf archive-name.tar /home

63. tar -xvf archive-name.tar

64. Shell:>grep Samuel /etc/passwd (grep Samuel details in /etc/passwd)

65. Shell:>find / -name /etc

66. Shell:>ps -ef | grep init

67. Shell:>kill -9 <pid>

68. Shell:>rm –I /home/kumar/*temp.txt (prompt before removal)

69. Shell:>rm –rf /home/kumar (remove with force directory and its contents)

70. Shell:>lsof -i TCP:22(list open files for tcp on port 22 of any internet address matches ‘-I’)

71. Shell:>lsof -u samuel ( list open file for user samuel)

72. Shell:>lsof -i 4(list open file for IPv4)

73. Shell:>lsof -i 6 list open file for IPv6)

74. Shell:>lsof -i TCP:1-1024

//Shell:>lsof –u ^root kumar

75. Shell:>lsof -i -u samuel

76. Shell:>lsof -p ^I ,^2 (list open files for other processors than 1 and 2)

77. Shell:>kill -9 `losof -t -u samuel`

78. Shell:>find . -name temp

79. Shell:>find /home -iname procscript.py(find case insensitive match)

80. Shell:>find / -type d -name etc

81. Shell:>find / -type f -name passwd

82. Shell:>find / -type f -name '*.php'(find file name)

83. Shell:>sudo find / -type d -name '*.d' | less( find directory )

84. Shell:>history 5(last 5 commands typed)

-------------------- ---------------- -------------------------

System time information and details.

85. Shell:>cd /usr/share/zoneinfo (find the zones )

86. Shell:>cd /etc/sysconfig/clock( The file which time zone is set for the system)

87. Shell:>echo 'ZONE="Europe/Copenhagen"' (the display the zone of particular region)

To remove the current zonetime and set a new time zone

88. Shell:>rm -f /etc/localtime;(remove the current localtime)

89. Shell:> ln -s /usr/share/zoneinfo/Europe/Copenhagen /etc/localtime(set a symbolic link to zone according to the requirement)

//$ diff -s /etc/localtime /usr/share/zoneinfo/`cat ./etc/timezone`

90. Shell:>diff -s ./localtime /usr/share/zoneinfo/US/Michigan(Compare files line by line.)

//dpkg-reconfigure tzdata

-------------------- --------------------------------------

91. Shell:>tar -tvf sample1-org.tar.bz2

92. Shell:>tar -xvf sample-org.tar salefile.sh

Shell:>tar --extract --file=salefile.sh

93. Shell:>tar -zxvf sample-org.tar.gz salefile.sh

tar --extract --file=sample-org.tar.gz salefile.sh

94. Shell:>tar -jxvf sample-org.tar.bz2 salefile.sh ( ‘-j’ option for bzip2)

tar --extract --file=sample-org.tar.bz2 salefile.sh

95. Shell:>tar -xvf sample-org.tar 'salefile.sh' 'orderfile.sh'

96. Shell:>tar -zxvf sample-org.tar.gz 'salefile.sh' 'orderfile.sh' (‘-z’ required for gzip)

97. Shell:>tar –jxvf sample-org.tar.bz2 'salefile.sh' 'orderfile.sh'

98. Shell:>tar -xvf sample-org.tar --wildcards *.sh'

99. Shell:>tar -rvf sample-org.tar salefile.sh orderfiledir (add files to end of archive)

100. Shell:>tar xvfw sample-org.tar(not valid for bz2,gzip)(verify the archive after writing it)

101. Shell:>uname –a(linux distribution complete details)

102. Shell:>id(id under which u are operating as)

103. Shell:>hostname(hostname given to the IP address)

104. Shell:>useradd kumar

105. Shell:>chfn -f 'kumar t' kumar(chfn is used to change your finger information. This command will display four pieces of information that can be changed by chfn: your real name, your work room and phone, and your home phone in RHEL distribution.)

106. Shell:>usermod kumar

107. Shell:>/usr/sbin/usermod -p '$1$u??e?z?.....' kumar(use encrypted password)

108. Shell:>groupadd devteam

109. Shell:>groupmod

110. Shell:> cat /etc/passwd(password file)

111. Shell:>cat /etc/groups(group file)

112. Shell:>cat /etc/shadow (encrypted password file for users)

113. Shell:>/etc/shadow/g

114. Shell:>who

115. Shell:>whoami

116. Shell:>chmod u+x demo1(change add file permissions for demo1 file for user )

117. Shell:>chmod u-x demo(change remove file permissions for demo file for user )

118. Shell:>chmod a=rwx demo((change add file permissions for demo1 file for user )

119. Shell:>chmod 744 demo

120. Shell:>chown kumar:devteam changescript.sh

chown [OPTION]... [OWNER][:[GROUP]] FILE

121. shell:>chgrp -hR devteam /home/kumar/changedir

122. shell:>find . type d -exec chmod 755 {} \;

123. shell:>find . -type f -exec chmod 777 {} \;

124. shell:>find . -type d -name join* -exec chmod 777 {} \

125. ------- ------------- ----------------------

DNS

126. Shell:>Bind:bekerly intenet name system

127. Shell:>Dig (cat /etc/resolv.conf to IP address of known domain servers)Dig is Dns lookup utility)

128. Use a Specific DNS server Using dig @dnsserver. By default dig uses the DNS servers defined in your /etc/resolv.conf file. If you like to use a different DNS server to perform the query, specify it in the command line as @dnsserver

129. Shell:>cat /etc/hosts (list of computers on your network)

130. Shell:> cat /etc/nsswitch.conf ( watch the file to see exactly where the user passwords and other important information being persisted with different options of local file system , dns , database or anywhere else.Indicates the system where to look for host system related information.)

---- ---------------------------------- -------------------------------

131. MTA(mail transfer agents) sendmail, pop, Imap

132. MUA(mail user agent) Evolution(supported by Linux to draft a email) , mail, netscape

133. MDA(mail delivery agent)Lda, local,(procmail)

134. mail program is used to transfer mails

135. shell:>cd /var/spool/mail ( is where the sent or received mails are stored for each individual user in the directory as specified)

136. bashshell:>mail usrname < user.txt

137. shell:>cd etc/mail( mail program location)

138. sendmail.cf configuration file

139. sendmail.mc macro for sendmail.cf(sendmail.cf file for use with sendmail)

140. bashshell:>m4 etc/mail/sendmail.mc > /etc/mail/sendmail.cf (m4 is a macro processor . Process macros in FILEs. If no FILE or if FILE is ‘-’, standard input is read.) esyscmd() tells m4 to run the given shell command and insert its output at the given location.define command defines the macro. ifdef checks if a macro has been defined.

----------------- -----------------------------------------------------

webserver

141. shell:>/etc/rc.d/init.d/httpd start(location of httpd program location .httpd is the webserver program)

142. shell:> service httpd start

143. shell:>chkconfig --level 5 httpd on (automatically start the httpd daemon)

144. or

145. shell:>redhat-config-services(It is stated that you can get to Services via, System → Administration → Services or by typing system-config-services at shell. To install use yum install redhat-config-services to install with dependencies)

146. shell:> less /etc/httpd/conf/httpd.conf(configuration directory for httpd service)

The httpd.conf file has lot of information regarding serverroot, listening for traffic, addons to apache web server called modules, extended status is on or off, user and group running apache web server who is owner and which group can run, servername ,log files storing ,log files information format, document root, restricting access to webserver using <Directory “/”> saying allow or deny for the folder/directory, saying which host machine can access it ).Different options directives can be set such as Indexed, includes, FollowSymlinks, Exec CGI, Multi views, these option directives can be set as “all/none” with restricting access to them with “allow/deny” qualifiers. Allow override directive for directory by directory can be given in httpd.conf file to restrict access. “.htaccess “ file should be created in each directory in apache webserver directories based on access rights set using allow override directives on those directories in httpd.conf file .

Sample snippet : order deny , allow

Deny from all

Allow from www.***.com

And you can also restrict access to directories by setting passwords by using command such as

htppasswd –c webusers kumar( to add more users to file webusers password file use” htppasswd webusers Samuel”) . And the apache to use this kind of authentication to restrict access you should change the settings in conf file as “Auth Name SecretStuff

AuthType Basic

Auth User file <path-to>/webusers

Require valid-user “

147.     shell:> yum search system-config-services

148.     shell:>redhat-config-services (redhat graphical user

    interface tool can be  used to configuration service)

149. apache web server main packages(install all the below rpm’s to get the full functionality of apache web server)

httpd, httpd-devel(development tools for apache web server),redhat-config-httpd(Graphical user interface help to configure web server), httpd-manual(documentation for the web server where it is located in directory at /var/www/html/manual)

mod-perl(perl lanquage interperters for apache web server),mod-python(python lanquage interperters for apache web server),mod-ssl(for encryptions so you can use protocols like secure http),autoconf(to create scripts),automake(make files),libtool(generic library tools for apache)

------------------- ---------------------------------------------------

150. Shell:>mount nandgama.example.com:/misc/export /misc/local (this command mounts the nfs share with server:directory details to the local machine directory. This command is used as a set up for one time use only)

151. fstab(file system table configuration file)

example : /dev/mapper/VolGroup-lv_root / ext4 defaults,noatime,discard,acl,usrquota 1 1(a entry in fstab file to set user quota)

152. nfs –server configuration “/etc/exports” is a configuration file . Command is

“directory host(options) [host (options)]”

Option asnyc is write to cache memory and then once it is full then write /sync Is force write to disk ,“no_root_squash” options is set to specify no added privileges for client user on the server machine to make any changes to dir/files, options “ro/rw” is read only access and rw read write access.

153. This is the entry in Fstab to mount the /usr/local/bin on nfs server to local /bin directory everytime the system starts up. “server:/usr/local/bin /bin nfs rsize=8192,wsize=8192,timeo=14,intr”( the rsize and wsize of data the client and server exchange with each other)

154. webmin interface( GUI interface to add nfs sever share point to local file system, which is better to do to in correct format instead of using emacs or VI)

155. Bonnie,Bonnie++, IOzone benchmark testing tools( Benchmark programs used between nfs server and client to determine the rsize and wsize ) You want to experiment with rsize and wsize that works and is fast as possible

156. From the client machine type “time dd if=/dev/zero of= /mnt/home/testfile bs=16k count 16384” and time it and find the best time it works. “time” : times a transaction, “dd” convert and copy a file dd of option: write to file instead of stdout, dd if options reads from file instead of standard input, bs options reads and writes bytes at a time, count options count only BLOCKS input blocks. And after the above procedure read it back to the client machine “time dd if= /mnt/home/testfile of=/dev/null bs=16k”. Average the time and figure out the best rsize and wsize.

157. Shell:>mount –l(shows the mount including with labels)

158. rmp -ivh package_name(install options install, verbose and hash options)

159. rpm –qa vsftpd* (verify options,query all)

160. rpm -e vsftpd-2.0.5.10.e15(erase options/uninstall)

161. rpm -ivh --nodep --force (to force and install without any dependency check.

162. rpm -Uvh package_name(update it with option “-U”)

163. rpm –Fvh package_name(freshen it up with option “-F”)

164. rpm -qip /home/kumar/Desktop/teradata-connector-1.4.1-h(query about the connector with option “-q”, -p package, -I info.

165. [kumar@localhost ~]$ rpm -qpl /home/kumar/Desktop/teradata-connector-1.4.1-hadoop2.x.noarch.rpm(check the rpm file with more detailed info , -l list )

166. [kumar@localhost ~]$ rpm -qf /var/ftp/pub

166.(check the file is created in which step)

167. rpm –Va package_name (verify –V, -a all)

------------------- ----------------------------------------------------------------

168. monolithic vs modular kernel's(manolithic kernel has everything inside of it , where as modular you install the necessary modules based on the need)

169. Initial RAMDISK (iscis initiators)(initial ramdisk is a disk image in memory . Most of the usage of the initial ramdisk is because the hard disks are scsi disks , the scsi drivers are not part of initial load of kernels modules, so you need a initial ram disk to boot of the system properly with scsi drivers.

170. In symmetric multiple processors machines you should install SMP kernel version for compatibility.

New RHEL5 based kernel uses different flavors naming:

UP kernel is no longer provided
SMP kernel comes without any flavor (like old UP)
entnosplit kernel comes as PAE
enterprise kernel comes as ent
ovzkernel.x86_64 for 64-bit

171. to create a checksum for any file

shell:>md5sum <filename>

172. go to grub.conf and find the boot section and find the title to figure out how many kernel versions are installed on the machine.

173. Download the kernel in /tmp directory. And install the kernel with “RPM –i /temp/ kernel-2.4.19-3.i386.rpm”

174. To see which processor u have go to shell:> cat /proc/cpuinfo and see the details of the processor . Based on cpu info you must download the kernel and install the kernel. “uname –a ”

175. There are two distributions of kernels available one in source code and one in binary. The source code distribution of kernel version is denoted by kernel-****src.rpm and other with binary version which is compiled ready to install. The src distribution should be compiled to binary before installation.

176. shell:>cd /boot and do a listing ls command directory to see the different files such as config-***(configuration file) ,initrd-***(intial ram disk ), modules files , system map, actual kernel for each distribution of kernel installed on the machine.

177. To install a kernel you need the following package

RPM’s Kernel-source-*,

glibc-kernel-headers-*(library for kernel headers),

glibc-devel-*(developmental c library),

cpp-*(c language preprocessor),

binutils-*(binary configuration utilities),

gcc-*(gnu c compiler),

tcl-*(scripting lanquage needed to run binutil configuration utility to run ), tk-*(scripting cd lanquage needed to run binutil configuration utility to run ).

178. shell:> cd /usr/src/linux-2.*(change to this directory to run the kernel operations) . And under the /usr/src/linux-2.*/configs/ directory you will see configuration template files. Based on cpu of the machine sample kernel configuration template file needs to be edited to your requirement/customize

179. Configure a custom kernel procedure(we need source code, c language compiler, libraries.

In dir path /usr/src/linux-2.* Shell:>make mrpoper

make command is nothing but a script execution named "MakeFile" in which it goes to the particular section given and executes it. “Make mrproper” command execute and proceed with caution. if you did a custom configuration on your system then you find a configuration file .config on your system , copy it to seperate directory and change the file name to different name and load it back to the Linux directory /usr/src/linux-2.4/. Otherwise if you a run the command “Make mrproper” then that command will delete the old .config file in the directory /usr/src/linux-2.4 which is your custom configuration file for kernel.

To make a configuration file .config you can use “make config(text based tool)/”make menuconfig”(it is still a text based installation but you can make changes if you do something wrong in the process) commands. To use make menuconfig you need to have two more important packages installed on the system which are ncurses4-*, ncurses-devel-*

180. (Redhat recommended installation process method for new kernel.)Run bashshell>Make xconfig to configure your linux system which is recommended by red hat.After completion of this step run the below commands

make dep command will check for dependencies. Make clean will prepare the source tree for source compilation

Make bzImage to build the kernel. Run Make Modules to make modules . Make modules_install will install the modules. Make install will copy the new kernel to all the associated directories and will create the ramdisk file by running the scripts.) Run the above commands shown above one after another to install the kernel with correct configuration.

NOT NEEDED{genisoimage \\\mkisofs -o file.iso -b isolinux.bin -c boot . cat -no-emul-boot -boot-load-size 4 -boot-info-table -R -J -v -T isolinux
mkisofs is effectively a pre-mastering program to generate an ISO9660/JOLIET/HFS hybrid filesystem.

{ An emulator typically enables the host system to run software or use peripheral devices designed for the guest system)

( -o filename Specify the output file for the the ISO9660 filesystem image.

-b eltorito_boot_image Specifies the path and filename of the boot image to be used when making an El Torito bootable CD for x86 PCs.

–c boot_catalog Specifies the path and filename of the boot catalog, which is required for an El Torito bootable CD

-boot-load-size load_sectors Specifies the number of "virtual" (512-byte) sectors to load in no-emulation mode. The default is to load the entire boot file. Some BIOSes may have problems if this is not a multiple of 4.

-boot-info-table Specifies that a 56-byte table with information of the CD-ROM layout will be patched in at offset 8 in the boot file

- -R Generate SUSP and RR records using the Rock Ridge protocol to further describe the files on the ISO9660 filesystem.

- -J Generate Joliet directory records in addition to regular ISO9660 filenames.

-T Generate a file TRANS.TBL in each directory on the CD-ROM, which can be used on non-Rock Ridge-capable systems to help establish the correct filenames }

181. use kernel version ”bigmem " distribution for i686 machines with more than 4gb of memory

182. shell:>umount (to un mount a device)

183. bashshell:>/sbin/mkbootdisk 2.4.20-69(The iso option instructs mkbootdisk to make a bootable ISO image on the hard disk that can later be used with a CD writer to create a bootable CDROM . The boot floppy created by mkbootdisk contains five files: boot.msg, initrd.img, ldlinux.sys, syslinux.cfg and vmlinuz. vmlinuz is the compressed Linux kernel executable, and it is bootable.)The general syntax for mkbootdisk is

mkbootdisk [options] kernel

usage: mkbootdisk [--version] [--noprompt] [--device <devicefile>] [--verbose -v] [--iso] [--kernelargs

(ex: mkbootdisk --device /dev/fd1 2.0.31)2.0.31(kernel version) – - device (use device file)

183. An entry in shell:>cd /etc/fstab file {/etc/fstab(/dev/mapper/VolGroup-lv_root / ext4 defaults,noatime,discard,acl,usrquota 1 1)

184. shell:>quotacheck --help

185. shell:>quotacheck -avcm (-a :all, c:create new quota files, -m :do not remount filesystem read-only)

186. shell:> edquota <user-name>(ex: edquota kumar)

187. shell:> cat /etc/warnquota.conf(will go with warnquota script)

188. sudo(super user do)

189. shell:>cd /etc/sudoers | less (read the sudoers file and watch for sections of

user_Alias( Runas_Alias

Host_Alias

Cmnd_Alais

Next comes the main part: which users can run what software on which machines (the sudoers file can be shared between multiple systems).

Syntax: user MACHINE=COMMANDS

The COMMANDS section may have other options added to it.

Example :Allow root to run any commands anywhere

root ALL=(ALL) ALL

kumar ALL=(ALL) ALL

190. use visudo(to edit sudoers file)

191. the sudo entries will be /var/log/secure file . shell:> cat /var/log/secure

192. useradd –D (print or change default useradd configuration)

193. shell:> cd /etc/skel directory (where the default files will be stored such as bash_logout .bash_profile .bashrc .gnome2 .mozilla, so that whenever a new user is created these defaults are assaigned to every user )shell:> ls –la /etc/skel

193. shell:>usermod -p <passwd> siri

194. shell:>userdel –r kumar(remove home directory and mail spool of user kumar)

195. shell:>userdel kumar

196. IN /etc/passwd file where the system related login information is stored

197. In /etc/shadow file where the encrypted passwords for logins are stored

198. In /etc/group file where the group information is stored

199. rhel-config-users/system-config-users( A graphical user interface to add user operation and group operations in large)

200. [root@localhost etc]# find / -name system-config-users

Results of the above command(notice the graphical system-config-users is in different paths

/usr/bin/system-config-users

/usr/share/gnome/help/system-config-users

/usr/share/system-config-users

/usr/share/system-config-users/system-config-users

/usr/share/omf/system-config-users

/etc/pam.d/system-config-users

/etc/sysconfig/system-config-users

/etc/security/console.apps/system-config-users

201. webmin a progam to install on linux which can be used to administer the machine or cluster (a script to create users in batch)

202. shell:> cat /etc/bashrc (A file in which fucntions and aliases go in /etc/bashrc) System wide functions and aliases

203. shell: >cat /etc/profile(system wide env and startup programs,for login setup)

shell will run 1st /etc/profile

2nd local .bash_profile(when logged in . Local env go in here)

3rd in .bash_profile runs a .bashrc (.bashrc: User specific aliases and functions)

4th .bashrc runs a /etc/bashrc

Hint: It's better to create a custom.sh shell script in # /etc/profile.d/ to make custom changes to your environment, as this will prevent the need for merging in future updates.

----------- ------------------------------------------------------

204. shell:> fsck (file system check) (fsck on root and /boot is avoided at most times)

Usage: fsck.ext4 [-panyrcdfvtDFV] [-b superblock] [-B blocksize]

[-I inode_buffer_blocks] [-P process_inode_size]

[-l|-L bad_blocks_file] [-C fd] [-j external_journal]

[-E extended-options] device

205. shell:>fdisk(create partition)( is a advanced program used to experienced users to create a partition by specifying options on the command line while running it. It will create /erase the old one if any mistakes are done which will have a loss of data)

Usage: fdisk [options] <disk> change partition table

Fdisk [options] -l <disk> list partition table(s)

fdisk -s <partition> give partition size(s) in blocks

203. e2label(changing labels on partition) usage: e2label device [newlabel]

204. sync(save changes to disk)

205. shell:>ls /lib/modules(where all the modules of the kernel are present)

206. shell:> sbin/mkbootdisk(to make a boot partition command)

207. shell:> yum install dvd+rw-tools

> dvd+rw-format -force /dev/sr0 (to format a dvd)

cdrecord dev=/dev/sr0 blank=fast(wodim is used to record data or audio Compact Discs on an Orange Book CD-Recorder or to write DVD media on a DVD-Recorder. Usage : wodim [options] track1...trackn)

Usage: dvd+rw-format [-force[=full]] [-lead-out|-blank[=full]]

[-ssa[=none|default|max|XXXm]] /dev/dvd

208. shell:>/sbin/mkbootdisk --device /dev/sr0 2.6.32-431.29.2.el6.i686

(it check for file named 2.6.32* on the current directory and writes on device /dev/sr0 as boot disk)

209. If Linux is unable to boot for some errors , put the installation floppy inside the dvd device and do a rescue operation. Three options available in rescue mode when running by a disc to do a rescue operation

-select f5 or type linux rescue to go into rescue mode

once in rescue mode you have three options “continue”, ”readonly”, ”skip”.

The entire file system will be mounted under /mnt/SYSimage if you choose continue,read-only.

210. shell:>fdisk –l (gives the information of the mount filesystem on the machine)

211. shell:>e2fsck /dev/sda1(can be checked only when the file system is not mounted because the /dev/sda1 is the /boot partition.)

212. shell:>e2label /dev/sda1 /boot

213. shell:>Mount /dev/sda2 /tmpMount/

214. sync(run it three times recommended by redhat)( sync - flush file system buffers)

215. shell:>umount /tmpmount

---------------- -------------------------------------------------

216. RPM(redhat package manager)

i-install,e-erase,-U upgrade,-q query, -V verify

217. shell:>rpmbuild --rebuild webmin-1.110-1.src.rpm(To build a binary file from source file)

218. shell:>redhat-config-packages

219. redhat network,Erata,Schedule,rhn.redhat.com(one more way to do a update of packages on the system by using rhn.redhat.com(login into rhn.redhat.com and follow the links)

All updates and errata for Red Hat Enterprise Linux are available via Red Hat Network (RHN) and can be downloaded and installed with up2date. The up2date is a utility provided with Red Hat Enterprise Linux 4

rhn_register(To register your system with Red Hat Network (RHN)

To list all updates available for your system:up2date –l

To download and install all updates: up2date –U

To download only all updates(these are stored in

/var/spool/up2date directory): up2date -u –d

By default, all kernel updates are on the "Skip List"

and will not be installed. To force up2date to install

any kernel updates you need to include

 the -f option: up2date -uf

220. shell:>/root/rpmbuild/RPMS/noarch/webmin-1.801-1.noarch.rpm(noarch is no architecture distribution can be installed on any machine)

------- ----------------------------------------------------------

221: desktop environments distributions for Linux (Gnome, KDE)

222. Window Managers(clients are displayed ,moved, resized,.... functionality is taken care by window managers)

Metacity ,sawfish,windowmaker(for Gnome)

kwin(for kde)

222. display managers-initial login screen(the display managers where the windows are managed to display without any additional functionality)

gdm(gnome display manager),kdm ,Display managers(gdm,kdm)

223. /etc/gnome/ (path where gnome related files present), ~/.gnome where “.gnome2” is present at home of every user which is his customized desktop option.

224. display manager is a xClient, default Display manager is gdm, in path/etc/X11/perfdm shell script sets default as scripted, Display manager starts a session manager( xsm - X Session Manager )xsm session manager once it started it takes care of xClient. Gdm (usr/sbin/gdm) same with gdm with gnome.

225. startx will excute X windows on the console . where X windows is a child process of shell

226. (dm vs startx)

==gnome-control-center(in path /usr/bin/gnome-control-center)

==to change window manager go to ~/.bash_profile edit it with export

window_manager ="sawfish" and save it .

==switchdesk gnome , switchdesk kde(will switch between gnome and kde desktop environments)

==gnome-"press tab" twice(to find more gnome programs)

In the typical client/server relationship, the client application runs on the local machine and the server application can run on either the local machine or a remote machine (i.e., any other computer on the network). In X, however. this relationship is inverted in that each local machine contains X server software and can access X client programs that run either on the same machine or on a remote machine. This has the advantage of eliminating the need for the application programs to be aware of the details of the graphics cards, monitors and other hardware, thereby simplifying the development of such programs and facilitating their servicing of multiple users on the network simultaneously.

Any application program that runs in a GUI provided by X, which is virtually every GUI used on Linux and other Unix-like operating systems, is an X client. Thus OpenOffice, Apache, gedit, gFTP, the GIMP, rCalc and Xpdf are all X clients when used on such operating systems.

-------------------------- ----------------------------------------------------------- ----------------------------------

227. xFree86(is a command line tool for X windows)( XFree86 was an implementation of the X Window System.)

228. xclient -display <hostname:0.0> -geometry Xsize x Ysize, +X offset,+Y offset {using xclient command setting for the windown X SIZE and Y size(with –geomety option) for the hostname given in “–display” option}

229. run xf86config is a program that generates XF86Config file

230. XF86Config is the configuration file for xFree86

231. shell:>info XF86config

232. shell:>X & (to start X windows)

== X server(x server to work perfectly need x font server running) An X server is a program in the X Window System that runs on local machines (i.e., the computers used directly by users) and handles all access to the graphics cards, display screens and input devices (typically a keyboard and mouse) on those computers

233. X font server (xfs service), XF86config, fonts must exits in path /usr/x11R6/lib/X11/fonts

xfs [ -config configuration_file ] [ -daemon ] [ -droppriv ] [ -inetd ] [ -ls listen_socket ] [ -nodaemon ] [ -port tcp_port ] [ -user username ]

xfs is the X Window System font server. It supplies fonts to X Window System display servers. The server is usually run by a system administrator, and started via init (8) or inetd (8) . Users may also wish to start private font servers for specific sets of fonts.

234. GUI Tools to configure x windows is redhat-config-xfree86(old tool is Xconfiguration)

235. alternatively run xf86config in /etc/X11 (be careful)

236. apropos xf86config, apropos gnome(The apropos command displays a list of all topics in the man pages)

237. xwindows is working with desktop environment and a window manager to give the

look and feel

238. shell:>/sbin/init 3 (to run in runlevel 3)

239. After starting at logon page press control +alt +f1 to get text based logon screen

==xterm -display localhost:0.0 & (to display x client), and press Alt+f7 to get xterminal window in xsession.

==cd /usr/X11R6/bin(is where window programs are present)

240. xclock -display localhost:0.0 -geometry 50X50-0-0& (display xclock window program as with geometry option)

----------------------- ------------------------------------------------------- ----------------------------------------- ------------------------

For example, if you're using iptables, you could allow traffic on these ports and eth1 (the private network interface) with these commands:( iptables — administration tool for IPv4 packet filtering and NAT)

241. Shell:>sudo iptables -A INPUT -i eth1 -p udp -m multiport --dports 5404,5405,5406 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT

{-i: Name of an interface via which a packet was received (only forpackets entering the INPUT, FORWARD and PREROUTING chains), -A:append,-p:protocol, -m :match, -d:address, -j: This specifies the target of the rule; i.e., what to do if the packet matches it, --ctstate:satelist is a comma separated list of the connection states to match. Possible states are listed below}

Iptables is used to set up, maintain, and inspect the tables of IPv4 packet filter rules in the Linux kernel. Several different tables may be defined. Each table contains a number of built-in chains and may also contain user-defined chains. Each chain is a list of rules which can match a set of packets. Each rule specifies what to do with a packet that matches. This is called a‘target’, which may be a jump to a user-defined chain in the same ta-ble.

242. Shell:>sudo iptables -A OUTPUT -o eth1 -p udp -m multiport –s ports 5404,5405,5406 -m conntrack

--ctstate ESTABLISHED -j ACCEPT--match -m match extended match (may load extension)

--proto -p proto protocol: by number or name, eg. `tcp'

--source -s address [/mask][...] source specification

--destination -d address[/mask][...] destination specification

--in-interface -i input name[+] network interface name ([+] for wildcard)

--jump -j target target for rule (may load target extension)

--out-interface -o output name[+] network interface name ([+] for wildcard)

--------------------------- -------------------------------------------- -----------------------------------------------------------

243. Shell:>less /root/install.log ( the installation log entries in /root/install.log directory)

244. Shell:>kudzu is a hardware indentification package.

kudzu detects and configures new and/or changed hardware on a
system.

When started, kudzu detects the current hardware, and checks it against

a database stored in /etc/sysconfig/hwconf, if one exists. It then

determines if any hardware has been added or removed from the system.

If so, it gives the users the opportunity to configure any added hard-

ware, and unconfigure any removed hardware. It then updates the

database in /etc/sysconfig/hwconf.

If no previous database exists, kudzu attempts to determine what

devices have already been configured, by looking at /etc/modprobe.conf,

/etc/sysconfig/network-scripts/, and /etc/X11/xorg.conf.

245. Shell:>cd /var/log> cat dmesg(dmesg is a system startup log)

246. Shell:>dmesg | less (www.linuxgazette.com/issue59/nazario.html is a explanation of dmesg file verbose output)

247. Shell:>/usr/bin/setup is program called text mode setup utility which actually to run )( setup - A text mode system configuration tool)

248.  Shell:>authconfig-tui,system-config-firewall,ntsysv,

      timeconfig{(authconfig,  authconfig-tui  -  an  interface

      for configuring  system  authentication resources.

      Use system-config-authentication  GUI application or the

     command line options instead.)( ntsysv - simple interface

      for configuring runlevels)(timeconfig  provides  a

      simple method of configuring both /etc/syscon-fig/clock and

     /etc/localtime which is compatible with both the standard

     Red Hat startup scripts

     and timeconfig(8).)}

Authconfig

[options] {--update|--updateall|--test|--probe|--restorebackup

<name>|--savebackup <name>|--restorelastbackup}

ntsysv [--back] [--level <levels>]

timeconfig [--utc] [--arc] [--back] [--test] [timezone]

249. Shell:>system-config-authentication

250. Shell:>redhat-config-network

251. Shell:>redhat-config-services

------------------------- ----------------------------------------- --------------------------------------------------------------

252. Shell:>Disk Druid(to setup the partitions and raid setup)

==Do not do a raid set up for /boot partition because the /boot should be at a single partition only

==And for other software raid partitions do not set up a mount point . Do setup a mount point after the set up of raid.

253. Shell:>fdisk –L (text based implantation of raid)( Partition table manipulator for Linux)

fdisk [-uc] [-b sectorsize] [-C cyls] [-H heads] [-S sects] device

fdisk -l [-u] [device...]

fdisk -s partition...

fdisk -v

fdisk -h

==select RAID button on the partitioning window and do the necessary steps to attach

the above software raid member partiions in this step to create MAKE A RAID DEVICE working with a mount point

==Text based setup of raid and graphical setup of Raid(disk druid)

254. Shell:>LVM(logical volume Management) new to redhat 8.0,pv(physical volume, logical volume group, logical volumes to mount point with each volume.

==to do a logical volume from a physical volume.

==Using Disk Druid through graphical representation we can do physical volume to logical volume.

==lvm,pv,lv,lvg and so on.(logical volume management, physical volume, logical volume,logical volume group)

255. shell:>info lvm (location of lvm directory /etc/lvm/lvm.conf, and location lvm of specific for user $HOME/.lvm_history

256. shell:>lvextend -L +100 /dev/volume00/logvol00(to extend 100 megabytes)( lvextend: Add space to a logical volume

==Raid levels(0,1,5),Hardware raid, software raid, Lvm (www.sistina.com)

---------------------------------------- --------------------------------------------------------------------

network installation issues:

257. Network installation, kickstart, Grub, Lilo boot loader.

258. In path /images we will have bootnet.img file to do a installation through network.

259. client machine(minumum installation),server machine(make access to this machine from any client machine so that network installation is done without problems)

260. shell:>mount -t iso9660 /dev/cdrom /mnt/cdrom(To mount the cdrom to the server machine under /mnt/cdrom)

261. shell:>df -h( to report filesystem disk space usage)

262. shell:>mkdir /rhinstall , cp –av /mnt/cdrom /rhinstall ( copy the contents of installtaion disk cdrom to the directory on /rhinstall directory on server machine)

==go to /etc directory on the server machine and do some change to export file emacs /etc/exports

/rhinstall * (ro) and save the exports file.

263. shell:>mount -t iso9660 /dev/cdrom /mnt/cdrom (do the mount on client machine ),copy "bootnet.img" to the client machine by using "dd" command,

==dd if =/mnt/cdrom/bootnet.img of=/dev/fdo (use dd when ur copying .img files )

==put the floppy disc and start the machine it will prompt a boot:where you type linux askmethod

"boot:linux askmethod"

==nfs,ftp,http(enter client and server information to install linux after prompt of boot:linux askmethod"

264. shell:>kickstart (using kickstart method to automate the installation of linux installation), using ks.cfg to automate,(use anaconda-ks.cfg in /root, edit sample.ks to get to required, use ksconfig GUI to do the required in /usr/sbin)

==boot: linux ks=floppy"

==boot: linux ks=nfs:192.168.0.100:/ks/ks.cfg"

==boot: linux ks=ftp://192.168.0.100:/ks/ks.cfg"

==boot: linux ks=http://192.168.0.100:/ks/ks.cfg"

==more anaconda-ks.cfg ( to see different options)

==grub, lilo (linux loader)

----------------------------------------- --------------------------------------

265. Disk Druid only available at installation

266. shell:>DD more user friendly,fdisk, sfdisk,fdisk allows actions to be scripted.

==DD is only available at installation , but after installation you want to partition then you got to use fdisk.

==tab to move to next option available and space bar to select .

-------------------------------------------- ---------------------------------------- ---------------------------------------

267. Business perspective, user perspective, technical issues.(while installation of Linux)

==business & end user perspective, Main system or supplemental, Starting fresh or migrating?

------------------

Types of problems

268. Hardware, operating system (kernel),application software, configuration, user

269. Kernels are denoted by x.y.z (“y” should be even number instead of odd number) stable

release is a y with a even number.

270. change and check in /etc for system wide problems , .filename problems with user specific

problems.

==Plan of attack Observer problem first hand (Recreate the problem), and work on it, formulate a guess as to cause, attempt to fix (keep records), Go back to step1 until fixed.

==preventative measures buy supported hardware, stay away from untested software, and don’t overclock

271. shell:>cd /var/log/ ; /var/log:>cat boot.log, maillog, maillog.1(boot.log in location of path /var/log/boot.log)

272. shell:>cd /etc/ , shell:>cat rsyslog.conf | less (rsysconf in /etc directory)

Bugprirority:debug,info,notice,warning,warn(depreciated),err,error(depreciated),crit,alert,emerg,panic(depreciated)

273. shell:>cat syslog.conf | less

Check the output , and you find few lines about paths of log files locations.

==mail.* /var/log/maillog, ==authpriv* /var/log/secure, ==cron* /var/log/cron, ==*.emerg *(to display to everybody) , ==kern.* /dev/console,

==*.info;mail.non;authpriv.none;cron.none /var/log/messages

274. shell:>dmesg (displays log files recorded when starting )( dmesg - print or control the kernel ring buffer)

275. shell:>ifconfig etho (ifconfig - configure a network interface)

SYNOPSIS

ifconfig [interface]

ifconfig interface [aftype] options | address ...

{Ifconfig is used to configure the kernel-resident network interfaces. It is used at boot time to set up interfaces as necessary. After that, it is usually only needed when debugging or when system tuning is needed. If no arguments are given, ifconfig displays the status of the currently active interfaces.

276. shell:>route (specficies the routes used by eth0 and eth1)( route - show / manipulate the IP routing table)

277. Shell:>df /boot (report file system disk space usage)

278, shell:>du *, du / , du /boot(du - estimate file space usage)

279. shell:>fsck (check and repair a Linux file system)

fsck.ext4 [-panyrcdfvtDFV] [-b superblock] [-B blocksize]

[-I inode_buffer_blocks] [-P process_inode_size]

[-l|-L bad_blocks_file] [-C fd] [-j external_journal] [-E extended-options] device

280. shell:>lpq(lpq - show printer queue status)

281. Shell:>top (top - display Linux tasks)

top -hv | -abcHimMsS -d delay -n iterations [-u user | -U user] -p pid [,pid ...]

-a sort by memory usage

-b batch mode operation

-M detect memory units

-s secure mode operation

282. shell:>cd /etc/init.d ( where all process scripts are present)

283. lock files(sometimes when the process is killed by a kill -9 option then the process lock file will prevent the service to get restared with a restart command. SO in that case search the process lock file and delete it and restart the process. The lock file will get generated when the process starts.)

284. when the process is already running any configuration changes for it to take effect will have to get restarted. But if you are restarting when it is in use the users will get effected , so make changes to process conf file and use kill -1 <pid>where -1 option for kill is sighup.sighup is saying that when the process is again restarted anytime soon please apply the necessary configuraition change without killing it.

Use start and stop scripts in etc/init.d directory to proper stop and start scripts of processes.

285. shell:>kill -l ( to see all the signals of the kill command type kill –l)

286. shell:> ip (show / manipulate routing, devices, policy routing and tunnels)

ip [ OPTIONS ] OBJECT { COMMAND | help }

OBJECT := { link | addr | addrlabel | route | rule | neigh | tunnel | maddr | mroute | monitor } OPTIONS := { -V[ersion] | -s[tatistics] | -r[esolve] | -f[amily] { inet| inet6 | ipx | dnet | link } | -o[neline]}

287. Shell:>ip addr , shell:> ip link, shell:> ip -s link (For statistics use)

288. shell:>cancel (cancel cancels existing print jobs. The -a option will remove all jobs from the specified destination, check these commands lp , lpr, lprm,lpstat

--------------------- ------------------------------------------------------------------------

hadooop ambari administration

Wednesday, August 10, 2016

Important Linux commands