Linux Files and Directories

Basics

------

File permissions:

.--- file type

|       .--- permission (uuugggooo)

|      |   .--- acl flag

|      |  | .--- number of links

|      |  | |   .--- user

|      |  | |   |    .--- group

|      |  | |   |    |       .--- size

|      |  | |   |    |       |      .--- date of modification

|      |  | |   |    |       |      |         .--- file name

|      |  | |   |    |       |      |         |

-rwxr-xr-x. 1 root users     6 May  7 16:59 file

Locating Files

--------------

Using find:

locating and deleting files

# deletes w/o confirmation find / -name *garbage* -exec rm -f {} \; # includes files with whitespaces on filenames find -type f | xargs -d "\n" rm # its funny the latter method is much faster [~/test]_$ time find / -xdev -type f -exec ls -l {} \; &> /dev/null real    1m3.165s user    0m18.897s sys     0m44.464s [~/test]_$ [~/test]_$ time ls -l `find / -xdev -type f` &> /dev/null real    0m1.370s user    0m0.814s sys     0m0.592s [~/test]_$

File Permissions

----------------

Basic permissions:

chmod

4 - read

2 - write

1 - execute

chmod <+/-/=>

##makes a file or directory immutable

chattr +i

##makes a file or directory unimmutable

chattr -i

##checks if an immutable flag is set on a file/directory

lsattr

chmod +[permission] file_name  ##grants permission to all users

chown   ##changes ownership of a file or directory

chown -c   ##shows information that changed

chown -f   ##disregards error messages

chown -R   ##executes the command recursively down the tree

chown -v   #verbose mode

chown :   ##simultaenousy changes both owner and group

chgrp   ##changes group ownership

Special Permissions:

SETUID (Set User ID)	## sets an SetUID (SUID) to a file or script chmod 4 chmod u<+/->s setUID on file: - a program is executed with the file owner's permissions (rather than with the permissions of the user who executes it) other notes: - capital S means directory's group has no execute permission - small S means directory's group has execute permission
SETGID (Set Group ID)	## sets a SetGID (SGID) to a file or directory chmod 2 chmod g<+/->s setGID on file: - the effective group of an executing program is the file owner group setGID on directory: - a newly created file under it will inherit its group - a newly created directory under it will will inherit its group - if setGID was added to its permission, any files or directories under it will retain their original permission;thus, setGID's effect will only apply to newly created files and directories other notes: - capital S means directory's group has no execute permission - small S means directory's group has execute permission
SVTX (Sticky Bits)	## sets a sticky bit (SVTX) to a file or directory chmod 1 chmod +t chmod u<+/->t - primarily used for directories - sticky bits on a directory prevents users to delete,modify,or rename files belonging to other users in that directory - owner of files/directories inside can modify,delete, or rename them - directory owner is permitted to create,delete,rename files inside but not modify

## what is the trailing dot at the end of a file permission?

drwxr-xr-x. 3 surendra surendra 4096 2011-07-06 00:19 Videos

The dot is indicating that files/folders are set with some sort of SELinux permissions on them

Directory Stacking

------------------

pushd         goes to a target dir

popd        goes to previous dir

dirs        lists dirs on stack

example:

[bob@server-new ~]$ pushd dir1

~/dir1 ~

[bob@server-new dir1]$ pushd dir2

~/dir1/dir2 ~/dir1 ~

[bob@server-new dir2]$ pushd dir3

~/dir1/dir2/dir3 ~/dir1/dir2 ~/dir1 ~

[bob@server-new dir3]$ dirs

~/dir1/dir2/dir3 ~/dir1/dir2 ~/dir1 ~

[bob@server-new dir3]$ popd

~/dir1/dir2 ~/dir1 ~

[bob@server-new dir2]$ popd

~/dir1 ~

[bob@server-new dir1]$ popd

~

[bob@server-new ~]$ popd

-bash: popd: directory stack empty

[bob@server-new ~]$

Commands

--------

File movement	# copies/moves/list only unhidden files and files w/o spaces on # filenames cp -p *.* /tmp mv *.* /tmp ls *.*
Listing	# prints directory tree (better than ls -lR) tree

ls -l non_existing_file &> file_name  ## redirects error messages to a specific file

ls -al | tee [option] file_name  ## saves the output to a file

cat > file_name  ## allows you to type on the shell and redirect it afterwards to a text file after hitting CTRL + D

Tutorials

---------

Mass renaming	for f in awx*; do git mv "$f" "${f//awx_/}"; done   # removes "awx_" on all files with filename starting at "awx"
to delete null device	cd /dev rm "null 2>&1"
recovers deleted file	lsof | grep file_deleted.txt ls -l /proc/PID_in_lsof/fd/any_number cp /proc/PID_in_lsof/fd/any_number file_restored.txt
deletes several files in batch	rm -i `cat filelist.txt`
Listing Open Files/Processes	# lists files opened under a directory lsof +D /var/log # lists files currently opened by steve lsof -u steve # lists all filea opened by all users except steve lsof -u ^steve # lists processes using a specified port lsof -i:80 fuser -v -n tcp 80
Find all hidden files	find -iname '.*' -ls

Inodes and Blocks

-----------------

- inode is set during fileystem creation time

stat file_name  ##prints the details of a file (together with inode number)

ls -i file_name  ##list the inode of a file

cp -l soft_link file_name  ##copies the soft link file rather than the original file

mv   ##moves a file from one location to another

mv   ##renames a file

*when new_file_name has a "/" at the beginning, file becomes invisible

du -h *  ##used to checks the system's block size (when 0, file is completely empty)

/dev/null  ##When written to, it discards all data

/dev/zeros  ##When read, it returns all zeros

/dev/tty  ##When accessed, it is redirected to the actual controlling device /dev/ttyx for this program

##to redirect message to a user using tty

~rell@sysx$ tty

/dev/pts/1

~rell@sysx$ echo hi /dev/pts/1

hi /dev/pts/1

~rell@sysx$

/dev/hda1  ##This refers to the first partition on the first IDE hard drive. Additional partitions are numbered /dev/hda2, /dev/hda3, etc. The second IDE harddrive is   /dev/hdb. This partition naming schemeallows direct access to any partition on any drive without any file-manager involvement

/dev/ram  ##Makes high memory look like a hard disk (for rescue)

ULIMIT

------

ulimit - max processes a user can open at a time

##to display current ulimit values

ulimit -a

##displays hard limit (max)

ulimit -Hn

##displays soft limit (warning level)

ulimit -Sn

##displays kernel max number of files

cat /proc/sys/fs/file-max

##displays number of currently open files

cat /proc/sys/fs/file-nr

note: commands above displays the ulimit values for the current user profile

UMASK

-----

## prints the currrent umask value

umask

## temporarily changes the current umask (to make changes permanent, append line to ~/.bashrc or /etc/profile. changes will apply upon next login)

umask xxx

## calculating permissions for files

666 -

## calculating permissions for directories

777 -

## umask table

Octal value        Permission

0                      ---

1                      --x

2                      -w-

3                      -wx

4                      r--

5                      r-x

6                      rw-

7                      rwx

*umask octal value for execute is by default turned off. To change execute rights, use chmod command.

File Conversions

----------------

iconv -f UTF-8 -t ASCII -c > --> converts UTF-8 to ASCII format

File Descriptors

----------------

##displays maximum number of system wide open file descriptors

cat /proc/sys/fs/file-max

##temporarily alters the value of open file descriptors

echo > /proc/sys/fs/file-max

sysctl -w fs.file-max=

##to apply changes permanently, add the following line in /etc/sysctl.conf

fs.file-max=

note: users need to logout and login again for the changes to take effect or issue the command: sysctl -p

/etc/security/limits.conf  ##you can set user ulimits here

syntax:

example:

merrell hard nofile 4096

Using "find" to search for files

--------------------------------

basics	##this prints full path names find /home/merrell -name file* ##this prints only file names find . -name file*
finding by size	##finds and list files in a simple format find -xdev -size [-|+][suffix] find -xdev -size [-|+][suffix] -print ##finds and list files in a long list format find -xdev -size [-|+][suffix] -exec ls -l {} \; find -xdev -size [-|+][suffix] | xargs ls -l
finding by access and modification time	##files that are accessed n minutes ago find -amin [-|+] ##files that are accessed n days ago find -atime [-|+] ##files that are modified n minutes ago find -mmin [-|+] ##files that are greater than n days old find -mtime + ##files that are less than n days old find -mtime - ##files that are exactly n days old find -mtime ##finds files newer than a specific file find /var -anewer thisfile find . -type f -printf "%T@ %p\n" --> finds newest files
finding by file details	##based on file system type find -fstype ##based on group id find -gid ##based on group name find -group ##based on inode number find -inum ##based on file type find -type -d  -->directory -f -->regular file -b -->block ##based on file permission find -perm ##based on file name find -name find -name -print ##returns all files with the following link numbers find -links ##finds files/directories belonging to a specific user find -user   notes: - if no path is specified, find will search on the current directory - size format may be written as +1G, +100000, etc.. examples: find / -xdev -size +1G find / -xdev -size -1000000 -exec ls -l {} \;
adding actions	find -name -exec {} \; find -name | xargs
options	-print -->ignores spaces in filenames -xdev  -->doesn't look under /proc filesystem which doesn't contain real files
by permission	find / -type f -perm 600      # finds all files with "644" permissions find . -type f -perm /222     # finds all files with atleast 1 "write" bit set
negating options	find . ! -perm 664 find . \! -type d         # escapes ! on some shells find /tmp -not -type f    # -not is same as !
by timestamp	find /dir -type f -printf '%T+ %p\n' | sort | head -n 1  # searches old file in a directory
others	# compressed logs before today        find . -name "*log*" -atime +1 -exec gzip -v {} \; http://alvinalexander.com/unix/edu/examples/find.shtml # provides non-xero exit status when no match found        find /tmp -type f -mtime +10 | egrep '.*' http://serverfault.com/questions/225798/find-command-enable-exit-status-to-be-diff-then-0-when-not-match # prints absolute path        find $(pwd) -type f

Using "locate"

--------------

locate -d database_file command_or_file  ##locates a command using mlocate database

locate --database=database_file command_or_file

locate command_or_file

locate -q command_or_file  ##ignores error messages

locate -i command_or_file  ##ignores casing

slocate          

*/var/lib/mlocate/mlocate.db  ##database file for locate command

updatedb  ##updates slocate/locate database

updatedb --netpaths=' ...'

updatedb -localpaths=' ….'

updatedb -f   ##exclude a directory in the update process

updatedb --output=  ##changes the destination database of the slocate/mlocate/locate command

updatedb -U   ##updates the slocate database startign from a certain path

updatedb -f   ##excludes filesystem type from updates

updatedb -v   ##verbose mode, displays the names of all relevant files as the slocate database is updated

apropos   ##lists manual pages of a command

makewhatis  ##database that stores information reg linux commands

XARGS vs EXEC

-------------

-exec will work like this..

grep "rGEO" a.txt

grep "rGEO" b.txt

however xargs will work like..

grep "rGEO" a.txt b.txt.

- xargs is faster most of the time

- use xargs in shell scripts to have faster response time

using xargs with filename with spaces        [root@ftp01 netops]# find test_dir/A/ -type f -mtime +90 -print0 | xargs -r -0 ls -lt

-rw-r--r-- 1 root root 0 Jan  1  2013 test_dir/A/file name with spaces

[root@ftp01 netops]#

--- XARGS ---

xargs: ls: terminated by signal 13

--> output of similar commands to find /home/ -type f | xargs -r ls -lt| head -1

--> signal 13 means something is written to a pipe where nothing is read from anymore

--> safe to ignore

Links

-----

*** what is a SOFT link? ***

- a pointer to a file or directory (like windows shortcuts)

- it has a different inode

- if you delete it, original file will remain

- it can cross different types of filesystems

commands in creating a soft link        ln -s

ln -s /path/to/

how to check if a file is a soft link?        [root@secdpdevdb01 ~]# ls -li soft_link_to_apple

 125973700 lrwxrwxrwx   1 root     root           5 Jan 27 14:12 soft_link_to_apple -> apple

[root@secdpdevdb01 ~]#

** permissions are 777

** there is "l" at the beginning of the permission

** there is a pointer arrow "->" at the end

examples of relative and absolute path on soft links        lrwxrwxrwx. 1 demo tutorial   15 May 17 21:43 mylink -> ../../thismonth

lrwxrwxrwx. 1 demo tutorial   20 May 17 21:48 mylink2 -> /home/demo/thismonth

*** what is a HARD link? ***

- a pointer to a file (cannot use in directories)

- it has same inode with the original file

- if you delete it, original file will remain

- it cannot cross different types of filesystems

commands in creating a hard link        ln

ln /path/to/

# returns the path referenced by a symlink

readlink

Access Lists (ACLs)

-------------------

- The filesystem containing the directory or file must support ACL in order for ACL to work

  -> you can check if FS has ACL support by "cat /etc/fstab"

  -> you can see "acl" on column 4

  -> to mount a filesystem w/ ACL support: # mount -t ext3 -o acl /dev/VolGroup00/LogVol02 /work