Tuesday, August 7, 2018

Linux Files and Directories


Basics
------


File permissions:

.--- file type
|       .--- permission (uuugggooo)
|      |   .--- acl flag
|      |  | .--- number of links
|      |  | |   .--- user
|      |  | |   |    .--- group
|      |  | |   |    |       .--- size
|      |  | |   |    |       |      .--- date of modification
|      |  | |   |    |       |      |         .--- file name
|      |  | |   |    |       |      |         |
-rwxr-xr-x. 1 root users     6 May  7 16:59 file


Locating Files
--------------

Using find:

locating and deleting files
# deletes w/o confirmation

find / -name *garbage* -exec rm -f {} \;

# includes files with whitespaces on filenames

find -type f | xargs -d "\n" rm

# its funny the latter method is much faster

[~/test]_$ time find / -xdev -type f -exec ls -l {} \; &> /dev/null
real    1m3.165s
user    0m18.897s
sys     0m44.464s
[~/test]_$
[~/test]_$ time ls -l `find / -xdev -type f` &> /dev/null
real    0m1.370s
user    0m0.814s
sys     0m0.592s
[~/test]_$

File Permissions
----------------

Basic permissions:

chmod
4 - read
2 - write
1 - execute
chmod <+/-/=>

##makes a file or directory immutable
chattr +i

##makes a file or directory unimmutable
chattr -i

##checks if an immutable flag is set on a file/directory
lsattr

chmod +[permission] file_name  ##grants permission to all users

chown   ##changes ownership of a file or directory
chown -c   ##shows information that changed
chown -f   ##disregards error messages
chown -R   ##executes the command recursively down the tree
chown -v   #verbose mode
chown :   ##simultaenousy changes both owner and group

chgrp   ##changes group ownership

Special Permissions:

SETUID (Set User ID)
## sets an SetUID (SUID) to a file or script
chmod 4
chmod u<+/->s

setUID on file:
- a program is executed with the file owner's permissions (rather than with the permissions of the user who executes it)

other notes:
- capital S means directory's group has no execute permission
- small S means directory's group has execute permission
SETGID (Set Group ID)
## sets a SetGID (SGID) to a file or directory
chmod 2
chmod g<+/->s

setGID on file:
- the effective group of an executing program is the file owner group

setGID on directory:
- a newly created file under it will inherit its group
- a newly created directory under it will will inherit its group
- if setGID was added to its permission, any files or directories under it will retain their original permission;thus, setGID's effect will only apply to newly created files and directories

other notes:
- capital S means directory's group has no execute permission
- small S means directory's group has execute permission
SVTX (Sticky Bits)
## sets a sticky bit (SVTX) to a file or directory
chmod 1
chmod +t
chmod u<+/->t

- primarily used for directories
- sticky bits on a directory prevents users to delete,modify,or rename files belonging to other users in that directory
- owner of files/directories inside can modify,delete, or rename them
- directory owner is permitted to create,delete,rename files inside but not modify


## what is the trailing dot at the end of a file permission?
drwxr-xr-x. 3 surendra surendra 4096 2011-07-06 00:19 Videos
The dot is indicating that files/folders are set with some sort of SELinux permissions on them


Directory Stacking
------------------

pushd         goes to a target dir
popd        goes to previous dir
dirs        lists dirs on stack

example:

[bob@server-new ~]$ pushd dir1
~/dir1 ~
[bob@server-new dir1]$ pushd dir2
~/dir1/dir2 ~/dir1 ~
[bob@server-new dir2]$ pushd dir3
~/dir1/dir2/dir3 ~/dir1/dir2 ~/dir1 ~
[bob@server-new dir3]$ dirs
~/dir1/dir2/dir3 ~/dir1/dir2 ~/dir1 ~
[bob@server-new dir3]$ popd
~/dir1/dir2 ~/dir1 ~
[bob@server-new dir2]$ popd
~/dir1 ~
[bob@server-new dir1]$ popd
~
[bob@server-new ~]$ popd
-bash: popd: directory stack empty
[bob@server-new ~]$

Commands
--------

File movement
# copies/moves/list only unhidden files and files w/o spaces on
# filenames
cp -p *.* /tmp
mv *.* /tmp
ls *.*
Listing
# prints directory tree (better than ls -lR)
tree

ls -l non_existing_file &> file_name  ## redirects error messages to a specific file
ls -al | tee [option] file_name  ## saves the output to a file
cat > file_name  ## allows you to type on the shell and redirect it afterwards to a text file after hitting CTRL + D

Tutorials
---------

Mass renaming
for f in awx*; do git mv "$f" "${f//awx_/}"; done
  # removes "awx_" on all files with filename starting at "awx"
to delete null device
cd /dev
rm "null 2>&1"
recovers deleted file
lsof | grep file_deleted.txt
ls -l /proc/PID_in_lsof/fd/any_number
cp /proc/PID_in_lsof/fd/any_number file_restored.txt
deletes several files in batch
rm -i `cat filelist.txt`
Listing Open Files/Processes

# lists files opened under a directory
lsof +D /var/log

# lists files currently opened by steve
lsof -u steve

# lists all filea opened by all users except steve
lsof -u ^steve

# lists processes using a specified port
lsof -i:80
fuser -v -n tcp 80
Find all hidden files
find -iname '.*' -ls

Inodes and Blocks
-----------------

- inode is set during fileystem creation time

stat file_name  ##prints the details of a file (together with inode number)
ls -i file_name  ##list the inode of a file
cp -l soft_link file_name  ##copies the soft link file rather than the original file

mv   ##moves a file from one location to another
mv   ##renames a file
*when new_file_name has a "/" at the beginning, file becomes invisible

du -h *  ##used to checks the system's block size (when 0, file is completely empty)

/dev/null  ##When written to, it discards all data
/dev/zeros  ##When read, it returns all zeros
/dev/tty  ##When accessed, it is redirected to the actual controlling device /dev/ttyx for this program

##to redirect message to a user using tty
~rell@sysx$ tty
/dev/pts/1
~rell@sysx$ echo hi /dev/pts/1
hi /dev/pts/1
~rell@sysx$

/dev/hda1  ##This refers to the first partition on the first IDE hard drive. Additional partitions are numbered /dev/hda2, /dev/hda3, etc. The second IDE harddrive is   /dev/hdb. This partition naming schemeallows direct access to any partition on any drive without any file-manager involvement
/dev/ram  ##Makes high memory look like a hard disk (for rescue)

ULIMIT
------

ulimit - max processes a user can open at a time

##to display current ulimit values
ulimit -a

##displays hard limit (max)
ulimit -Hn

##displays soft limit (warning level)
ulimit -Sn

##displays kernel max number of files
cat /proc/sys/fs/file-max

##displays number of currently open files
cat /proc/sys/fs/file-nr

note: commands above displays the ulimit values for the current user profile

UMASK
-----

## prints the currrent umask value
umask

## temporarily changes the current umask (to make changes permanent, append line to ~/.bashrc or /etc/profile. changes will apply upon next login)
umask xxx

## calculating permissions for files
666 -

## calculating permissions for directories
777 -

## umask table
Octal value        Permission
0                      ---
1                      --x
2                      -w-
3                      -wx
4                      r--
5                      r-x
6                      rw-
7                      rwx

*umask octal value for execute is by default turned off. To change execute rights, use chmod command.

File Conversions
----------------

iconv -f UTF-8 -t ASCII -c > --> converts UTF-8 to ASCII format

File Descriptors
----------------

##displays maximum number of system wide open file descriptors
cat /proc/sys/fs/file-max

##temporarily alters the value of open file descriptors
echo > /proc/sys/fs/file-max
sysctl -w fs.file-max=

##to apply changes permanently, add the following line in /etc/sysctl.conf
fs.file-max=
note: users need to logout and login again for the changes to take effect or issue the command: sysctl -p


/etc/security/limits.conf  ##you can set user ulimits here
syntax:
example:
merrell hard nofile 4096

Using "find" to search for files
--------------------------------

basics
##this prints full path names
find /home/merrell -name file*

##this prints only file names
find . -name file*
finding by size
##finds and list files in a simple format
find -xdev -size [-|+][suffix]
find -xdev -size [-|+][suffix] -print

##finds and list files in a long list format
find -xdev -size [-|+][suffix] -exec ls -l {} \;
find -xdev -size [-|+][suffix] | xargs ls -l
finding by access
and modification time
##files that are accessed n minutes ago
find -amin [-|+]

##files that are accessed n days ago
find -atime [-|+]

##files that are modified n minutes ago
find -mmin [-|+]

##files that are greater than n days old
find -mtime +

##files that are less than n days old
find -mtime -

##files that are exactly n days old
find -mtime

##finds files newer than a specific file
find /var -anewer thisfile

find . -type f -printf "%T@ %p\n" --> finds newest files
finding by file details
##based on file system type
find -fstype

##based on group id
find -gid

##based on group name
find -group

##based on inode number
find -inum

##based on file type
find -type
-d  -->directory
-f -->regular file
-b -->block

##based on file permission
find -perm

##based on file name
find -name
find -name -print

##returns all files with the following link numbers
find -links

##finds files/directories belonging to a specific user
find -user  

notes:
- if no path is specified, find will search on the current directory
- size format may be written as +1G, +100000, etc..
examples:
find / -xdev -size +1G
find / -xdev -size -1000000 -exec ls -l {} \;
adding actions
find -name -exec {} \;
find -name | xargs
options
-print -->ignores spaces in filenames
-xdev  -->doesn't look under /proc filesystem which doesn't contain real files
by permission
find / -type f -perm 600      # finds all files with "644" permissions
find . -type f -perm /222     # finds all files with atleast 1 "write" bit set
negating options
find . ! -perm 664
find . \! -type d         # escapes ! on some shells
find /tmp -not -type f    # -not is same as !
by timestamp
find /dir -type f -printf '%T+ %p\n' | sort | head -n 1  # searches old file in a directory
others
# compressed logs before today        find . -name "*log*" -atime +1 -exec gzip -v {} \;

# provides non-xero exit status when no match found        find /tmp -type f -mtime +10 | egrep '.*'
# prints absolute path        find $(pwd) -type f

Using "locate"
--------------

locate -d database_file command_or_file  ##locates a command using mlocate database
locate --database=database_file command_or_file
locate command_or_file
locate -q command_or_file  ##ignores error messages
locate -i command_or_file  ##ignores casing

slocate          

*/var/lib/mlocate/mlocate.db  ##database file for locate command

updatedb  ##updates slocate/locate database
updatedb --netpaths=' ...'
updatedb -localpaths=' ….'
updatedb -f   ##exclude a directory in the update process
updatedb --output=  ##changes the destination database of the slocate/mlocate/locate command
updatedb -U   ##updates the slocate database startign from a certain path
updatedb -f   ##excludes filesystem type from updates
updatedb -v   ##verbose mode, displays the names of all relevant files as the slocate database is updated

apropos   ##lists manual pages of a command

makewhatis  ##database that stores information reg linux commands

XARGS vs EXEC
-------------

-exec will work like this..
grep "rGEO" a.txt
grep "rGEO" b.txt

however xargs will work like..
grep "rGEO" a.txt b.txt.

- xargs is faster most of the time
- use xargs in shell scripts to have faster response time

using xargs with filename with spaces        [root@ftp01 netops]# find test_dir/A/ -type f -mtime +90 -print0 | xargs -r -0 ls -lt
-rw-r--r-- 1 root root 0 Jan  1  2013 test_dir/A/file name with spaces
[root@ftp01 netops]#

--- XARGS ---

xargs: ls: terminated by signal 13
--> output of similar commands to find /home/ -type f | xargs -r ls -lt| head -1
--> signal 13 means something is written to a pipe where nothing is read from anymore
--> safe to ignore

Links
-----

*** what is a SOFT link? ***

- a pointer to a file or directory (like windows shortcuts)
- it has a different inode
- if you delete it, original file will remain
- it can cross different types of filesystems

commands in creating a soft link        ln -s
ln -s /path/to/
how to check if a file is a soft link?        [root@secdpdevdb01 ~]# ls -li soft_link_to_apple
 125973700 lrwxrwxrwx   1 root     root           5 Jan 27 14:12 soft_link_to_apple -> apple
[root@secdpdevdb01 ~]#

** permissions are 777
** there is "l" at the beginning of the permission
** there is a pointer arrow "->" at the end
examples of relative and absolute path on soft links        lrwxrwxrwx. 1 demo tutorial   15 May 17 21:43 mylink -> ../../thismonth
lrwxrwxrwx. 1 demo tutorial   20 May 17 21:48 mylink2 -> /home/demo/thismonth

*** what is a HARD link? ***

- a pointer to a file (cannot use in directories)
- it has same inode with the original file
- if you delete it, original file will remain
- it cannot cross different types of filesystems

commands in creating a hard link        ln
ln /path/to/


# returns the path referenced by a symlink
readlink

Access Lists (ACLs)
-------------------

- The filesystem containing the directory or file must support ACL in order for ACL to work
  -> you can check if FS has ACL support by "cat /etc/fstab"
  -> you can see "acl" on column 4
  -> to mount a filesystem w/ ACL support: # mount -t ext3 -o acl /dev/VolGroup00/LogVol02 /work

No comments:

Post a Comment