GitHub - trimstray/sysadmin-interview-questions: A collection of Sysadmin Interv...

README.md

"A great Admin doesn't need to know everything, but they should be able to come up with amazing solutions to impossible projects." - cwheeler33 (ServerFault)

Table of Contents

• Introduction
  • Simple Questions - 10 questions.
• General Knowledge
  • Junior Sysadmin - 45 questions.
  • Regular Sysadmin - 62 questions.
  • Senior Sysadmin - 64 questions.
• Secret Knowledge
  • Guru Sysadmin - 12 questions.

Introduction

?Simple Questions

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

General Knowledge

?Junior Sysadmin

• Redhat Enterprise Linux
  
• Fedora Linux
  
• CentOS Linux
  
• Debian Linux
  
• Ubuntu Linux
  
• Suse Enterprise Linux
  
• Slackware Linux

UNIX - only big companies are allowed to use the UNIX copyright and name. IBM AIX, Sun Solaris, and HP-UX all are UNIX operating systems. Most UNIX operating systems are commercial in nature.

• UNIX operating systems are considered as a complete OS as everything come from a single vendor.
  
• Most UNIX like operating systems are not free.
  
• UNIX operating systems comes with its own firewall products.
  
• UNIX supports file systems like jfs, gpfs (AIX), jfs, gpfs (HP-UX), jfs, gpfs (Solaris).
  

Linux is a Unix clone. But if you consider Portable Operating System Interface (POSIX) standards then Linux can be considered as UNIX.

• Linux is Free. You can download it from the Internet or redistribute it under GNU licenses.
  
• All Linux distributions include GUI system, GNU utilities, installation & management tools, GNU c/c++ Compilers, Editors (vi), and various applications like OpenOffice, Firefox.
  
• Linux comes with open source Netfilter and IPTables based firewall tool to protect your server and desktop from the crackers and hackers.
  
• By default, Linux supports and use ext3 or ext4 file systems.

• GUI, file, and windows managers (KDE, Gnome)
  
• Shells (ksh, csh, bash)
  
• Various office applications such as OpenOffice.org
  
• Development tools like perl, php, python, GNU c/c++ compilers
  
• Posix interface
  

Linux operating system is consist of 3 components which are as below:

• Kernel: Linux is a monolithic kernel that is free and open source
  software that is responsible for managing hardware resources for the users.
• System Library: System Library plays a vital role because application programs access Kernels feature using system library.
  
• System Utility: System Utility performs specific and individual level tasks.

CLI is an acronym for Command Line Interface. We have to provide the information to the computer so that it can perform the function accordingly. In Linux, CLI is the interface that provides the user an interface so that user can type the commands and it complete the tasks. CLI is very easy to use, but it should be typed very precisely.

BASH as my favorite. It’s really a preferential kind of thing, where I love the syntax and it just "clicks" for me. The input/output redirection syntax (>>, << 2>&1, 2>, 1>, etc) is similar to C++ which makes it click even easier in my mind.

I also like ZSH shell because is much much more customizable than BASH. It has great Oh-My-Zsh framework, powerful context based tab completion, pattern matching/globbing on alien steroids, loadable modules and more.

The root account is like a systems administrator account, and allows you full control of the system (is the most privileged account on a Unix system). The "root" account has no security restrictions imposed upon it. This means it is easy to perform administrative duties without hassle. Root always has a UID of zero.

For more information, including each login of a username, the terminal users are attached to, the date and time that login began, and possibly the computer from which they are making the connection, enter:

who

For extensive information, including username, terminal, IP number of the source computer, the time the login began, any idle time, process CPU cycles, job CPU cycles, and the currently running command, enter:

w

UID and GID are numeric identifiers for users and groups. Unix-like operating systems identify a user within the kernel by a value called a user identifier, often abbreviated to user ID or UID. The UID, along with the group identifier (GID) and other access control criteria, is used to determine which system resources a user can access.

dmesg command is used to review boot messages. This command will display system messages contained in the kernel ring buffer. We can use this command immediately after booting to see boot messages. A ring buffer is a buffer of fixed size for which any new data added to it overwrites the oldest data in it.

Swap space is used when the amount of physical memory (RAM) is full. If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space. While swap space can help machines with a small amount of RAM, it should not be considered a replacement for more RAM. Swap space is located on hard drives, which have a slower access time than physical memory.

Using free and vmstat command we can display the physical and virtual memory statistics respectively. With the help of sar command we see the CPU utilization & other stats.

Is the average system load calculated over a given period of time of 1, 5 and 15 minutes. Lower numbers are better. Higher numbers represent a problem or an overloaded machine.

1. fdisk or gparted - create a new partition
   
2. mkfs - create a new filesystem

Journaling has a dedicated area in the file system, where all the changes are tracked. When the system crashes, the possibility of file system corruption is less because of journaling.

• Read: users can read the files or list the directory
  
• Write: users may write to the file or add new files to the directory
  
• Execute: users may run the file or lookup a specific file within a directory

Linux passwords are stored in the /etc/shadow file. They are salted and the algorithm being used depends on the particular distribution and is configurable.

To change all the directories e.g. to 755 (drwxr-xr-x):

To change all the files e.g. to 644 (-rw-r--r--):

It’s a fairly simple process, allowing you to direct data from one output to another. You can also use it to direct an output as an input to another process.

grep searches file patterns. If you are looking for a specific pattern in the output of another command, grep highlights the relevant lines. Use this grep command for searching log files, specific processes, and more.

• head: to check the starting of a file.
  
• tail: to check the ending of the file. It is the reverse of head command.
  
• cat: used to view, create, concatenate the files.
  
• more: used to display the text in the terminal window in pager form.
  
• less: used to view the text in the backward direction and also provides single line movement.

• SIGHUP - the SIGHUP signal is sent to a process when its controlling terminal is closed. It was originally designed to notify the process of a serial line drop (a hangup). Many daemons will reload their configuration files and reopen their logfiles instead of exiting when receiving this signal.
  
• SIGINT - the SIGINT signal is sent to a process by its controlling terminal when a user wishes to interrupt the process. This is typically initiated by pressing Ctrl+C, but on some systems, the "delete" character or "break" key can be used.
  
• SIGKILL - the SIGKILL signal is sent to a process to cause it to terminate immediately (kill). In contrast to SIGTERM and SIGINT, this signal cannot be caught or ignored, and the receiving process cannot perform any clean-up upon receiving this signal.
  
• SIGTERM - the SIGTERM signal is sent to a process to request its termination. Unlike the SIGKILL signal, it can be caught and interpreted or ignored by the process. This allows the process to perform nice termination releasing resources and saving state if appropriate. SIGINT is nearly identical to SIGTERM.

rm removes files and -rf are to options:

• -r remove directories and their contents recursively
  
• -f ignore nonexistent files, never prompt

tar tf archive.tgz
tar xf archive.tgz filename

To sync the contents of dir1 to dir2 on the same system, type:

rsync -av --progress --delete dir1/ dir2

• -a, --archive - archive mode
• --delete - delete extraneous files from dest dirs
• -v, --verbose - verbose mode (increase verbosity)
• --progress - show progress during transfer

cp filename{,.orig}

find / -type f -size +20M

su - invokes a login shell after switching the user. A login shell resets most environment variables, providing a clean base.

su just switches the user, providing a normal shell with an environment nearly the same as with the old user.

find / -mmin 60 -type f

1. You type api.example.com into the address bar of your browser.
2. The browser checks the cache for a DNS record to find the corresponding IP address of api.example.com.

• first, it checks the browser cache. The browser maintains a repository of DNS records for a fixed duration for websites you have previously visited. So, it is the first place to run a DNS query.
• second, the browser checks the OS cache. If it is not found in the browser cache, the browser would make a system call to your underlying computer OS to fetch the record since the OS also maintains a cache of DNS records.
• third, it checks the router cache. If it’s not found on your computer, the browser would communicate with the router that maintains its’ own cache of DNS records.
• fourth, it checks the ISP cache. If all steps fail, the browser would move on to the ISP. Your ISP maintains its’ own DNS server which includes a cache of DNS records which the browser would check with the last hope of finding your requested URL.

1. If the requested URL is not in the cache, ISP’s DNS server initiates a DNS query to find the IP address of the server that hosts api.example.com.

• the purpose of a DNS query is to search multiple DNS servers on the internet until it finds the correct IP address for the website. This type of search is called a recursive search since the search will continue repeatedly from DNS server to DNS server until it either finds the IP address we need or returns an error response saying it was unable to find it.
• for api.example.com, first, the DNS recursor will contact the root name server. The root name server will redirect it to .com domain name server. .com name server will redirect it to google.com name server. google.com name server will find the matching IP address for api.example.com in its’ DNS records and return it to your DNS recursor which will send it back to your browser.

1. Browser initiates a TCP connection with the server.

• once the browser receives the correct IP address it will build a connection with the server that matches IP address to transfer information. Browsers use internet protocols to build such connections.
• in order to transfer data packets between your computer(client) and the server, it is important to have a TCP connection established. This connection is established using a process called the TCP/IP three-way handshake.

1. The browser sends an HTTP request to the web server.

• this request will also contain additional information such as browser identification (User-Agent header), types of requests that it will accept (Accept header), and connection headers asking it to keep the TCP connection alive for additional requests. It will also pass information taken from cookies the browser has in store for this domain.

1. The server handles the request and sends back a response.

• the server contains a web server (i.e Apache, IIS) which receives the request from the browser and passes it to a request handler to read and generate a response.

1. The server sends out an HTTP response.

• the server response contains the web page you requested as well as the status code, compression type (Content-Encoding), how to cache the page (Cache-Control), any cookies to set, privacy information, etc.

1. The browser displays the HTML content (for HTML responses which is the most common).

Using the commands netstat -nr, route -n or ip route show we can see the default route and routing tables.

127.0.0.1 is the loopback Internet protocol (IP) address also referred to as the "localhost". The address is used to establish an IP connection to the same machine or computer being used by the end-user.

localhost resolves to the IP address 127.0.0.1, which is the most commonly used IPv4 loopback address, and to the IPv6 loopback address ::1. The name localhost is also a reserved top-level domain name, set aside to avoid confusion with the definition as a hostname.

# with host command:
host domain.com 8.8.8.8
# with dig command:
dig @9.9.9.9 google.com

# with telnet command:
telnet code42.example.com 5432
# with nc (netcat) command:
nc -vz code42.example.com 5432

Telnet uses most insecure method for communication. It sends data across the network in plain text format and anybody can easily find out the password using the network tool.

In the case of Telnet, these include the passing of login credentials in plain text, which means anyone running a sniffer on your network can find the information he needs to take control of a device in a few seconds by eavesdropping on a Telnet login session.

The main differences are: wget's major strong side compared to curl is its ability to download recursively. Wget is command line only. Curl supports FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, POP3, IMAP, SMTP, RTMP and RTSP.

Packet filtering is a firewall technique used to control network access by monitoring outgoing and incoming packets and allowing them to pass or halt based on the source and destination Internet Protocol (IP) addresses, protocols and ports.

A proxy server is a dedicated computer or a software system running on a computer that acts as an intermediary between an endpoint device, such as a computer, and another server from which a user or client is requesting a service.

Routers and Gateways are used to regulate network traffic between two or more separate networks. Gateways regulate traffic between two dissimilar networks, while routers regulate traffic between similar networks.

A default gateway serves as an access point or IP router that a networked computer uses to send information to a computer in another network or the internet. Default simply means that this gateway is used by default, unless an application specifies another gateway.

A gateway is a node (router) in a computer network, a key stopping point for data on its way to or from other networks. Thanks to gateways, we are able to communicate and send data back and forth.

DNS records are basically mapping files that tell the DNS server which IP address each domain is associated with, and how to handle requests sent to each domain. Some DNS records syntax that are commonly used in nearly all DNS record configurations are A, AAAA, CNAME, MX, PTR, NS, SOA, SRV, TXT, and NAPTR.

• SOA - A Start Of Authority
• A - Address Mapping records
• AAAA - IP Version 6 Address records
• CNAME - Canonical Name records
• MX - Mail exchanger record
• NS - Name Server records
• PTR - Reverse-lookup Pointer records

Whether you have a standard /24 VLAN for end users, a /30 for point-to-point links, or something in between and subnet that must contain up to 30 devices works out to be a /27 - or a subnet mask of 255.255.255.224.

• 1xx - Informational responses - communicates transfer protocol-level information
• 2xx - Success - indicates that the client’s request was accepted successfully
• 3xx - Redirection - indicates that the client must take some additional action in order to complete their request
• 4xx - Client side error - this category of error status codes points the finger at clients
• 5xx - Server side error - the server takes responsibility for these error status codes

It is a system that records changes to a file or set of files over time so that you can recall specific versions later. Version control systems consist of a central shared repository where teammates can commit changes to a file or set of file. Then you can mention the uses of version control.

Version control allows you to:

• revert files back to a previous state
• revert the entire project back to a previous state
• compare changes over time
• see who last modified something that might be causing a problem
• who introduced an issue and when

• git init - create a new local repository
• git commit -m "message" - commit changes to head
• git status - list the files you've added with git add and also commit any files you've changed since then
• git push origin master - send changes to the master branch of your remote repository

Security misconfiguration is a vulnerability when a device/application/network is configured in a way which can be exploited by an attacker to take advantage of it. This can be as simple as leaving the default username/password unchanged or too simple for device accounts etc.

?Regular Sysadmin

BIOS: Full form of BIOS is Basic Input or Output System that performs integrity checks and it will search and load and then it will execute the bootloader.

MBR: MBR means Master Boot Record. MBR contains the information regarding GRUB and executes and loads this bootloader.

GRUB: GRUB means Grand Unified Bootloader. In case, many kernel images are installed on your system then you can select which one you want to execute.

Kernel: Root file system is mounted by Kernel and executes the /sbin/init program.

Init: Init checks the file /etc/inittab and decides the run level. There are seven-run levels available from 0-6. It will identify the default init level and will load the program.

Runlevel programs: As per your default settings for the run level, the system will execute the programs.

toor is an "alternative" superuser account, where toor is root spelled backwards. It is intended to be used with a non-standard shell so the default shell for root does not need to change.

This is important as shells which are not part of the base distribution, but are instead installed from ports or packages, are installed in /usr/local/bin which, by default, resides on a different file system. If root's shell is located in /usr/local/bin and the file system containing /usr/local/bin) is not mounted, root will not be able to log in to fix a problem and will have to reboot into single-user mode in order to enter the path to a shell.

Some people use toor for day-to-day root tasks with a non-standard shell, leaving root, with a standard shell, for single-user mode or emergencies. By default, a user cannot log in using toor as it does not have a password, so log in as root and set a password for toor before using it to login.

By default, the main system log is /var/log/messages. This file contains all the messages and the script written by the user.

By default all scripts are saved in this file. This is the standard system log file, which contains messages from all system software, non-kernel boot issues, and messages that go to dmesg (is a system file that is written upon system boot).

/proc is a virtual file system that provides detailed information about kernel, hardware and running processes.

Since /proc contains virtual files, it is called virtual file system. These virtual files have unique qualities. Most of them are listed as zero bytes in size.

Virtual files such as /proc/interrupts, /proc/meminfo, /proc/mounts and /proc/partitions provide an up-to-the-moment glimpse of the system’s hardware. Others: /proc/filesystems file and the /proc/sys/ directory provide system configuration information and interfaces.

The problem with a load of 1.00 is that you have no headroom. In practice, many sysadmins will draw a line at 0.70:

The "Need to Look into it" Rule of Thumb: 0.70 If your load average is staying above > 0.70, it's time to investigate before things get worse.

The "Fix this now" Rule of Thumb: 1.00. If your load average stays above 1.00, find the problem and fix it now. Otherwise, you're going to get woken up in the middle of the night, and it's not going to be fun.

Rule of Thumb: 5.0. If your load average is above 5.00, you could be in serious trouble, your box is either hanging or slowing way down, and this will (inexplicably) happen in the worst possible time like in the middle of the night or when you're presenting at a conference. Don't let it get there.

On Linux and other Unix-like operating systems, new files are created with a default set of permissions. Specifically, a new file's permissions may be restricted in a specific way by applying a permissions "mask" called the umask. The umask command is used to set this mask, or to show you its current value.

Permanently change (set e.g. umask 02):

• ~/.profile
  
• ~/.bashrc
  
• ~/.zshrc
  
• ~/.cshrc

A symbolic link, also termed a soft link, is a special kind of file that points to another file, much like a shortcut in Windows or a Macintosh alias. Unlike a hard link, a symbolic link does not contain the data in the target file. It simply points to another entry somewhere in the file system.

a) Hardlink cannot be created for directories. Hard link can only be created for a file.

b) Softlink also termed a symbolic links or symlinks can link to a directory.

A Sticky bit is a permission bit that is set on a file or a directory that lets only the owner of the file/directory or the root user to delete or rename the file. No other user is given privileges to delete the file created by some other user.

To change the run level we have to edit the file /etc/inittab and change initdefault entry (id:5:initdefault:). Using init command we change the run level temporary like init 3, this command will move the system in runlevl 3.

Restart the system, type boot -s at the Boot: prompt to enter single-user mode.

At the question about the shell to use, hit Enter which will display a # prompt.

Enter mount -urw / to remount the root file system read/write, then run mount -a to remount all the file systems.

Run passwd root to change the root password then run exit to continue booting.

To set the kernel parameters in UNIX-like, first edit the file /etc/sysctl.conf after making the changes save the file and run the command sysctl -p, this command will make the changes permanently without rebooting the machine.

ext2

• ext2 stands for second extended file system.
  
• ext2 does not have journaling feature.
  
• on flash drives, usb drives, ext2 is recommended, as it doesn’t need to do the over head of journaling.
  
• maximum individual file size can be from 16 GB to 2 TB.
  
• overall ext2 file system size can be from 2 TB to 32 TB.
  

ext3

• ext3 stands for third extended file system.
  
• starting from Linux Kernel 2.4.15 ext3 was available.
  
• the main benefit of ext3 is that it allows journaling.
  
• maximum individual file size can be from 16 GB to 2 TB.
  
• overall ext3 file system size can be from 2 TB to 32 TB.
  
• you can convert a ext2 file system to ext3 file system directly (without backup/restore).
  

ext4

• ext4 stands for fourth extended file system.
  
• starting from Linux Kernel 2.6.19 ext4 was available.
  
• supports huge individual file size and overall file system size.
  
• maximum individual file size can be from 16 GB to 16 TB.
  
• overall maximum ext4 file system size is 1 EB (exabyte). 1 EB = 1024 PB (petabyte). 1 PB = 1024 TB (terabyte).
  
• directory can contain a maximum of 64,000 subdirectories (as opposed to 32,000 in ext3).
  
• you can also mount an existing ext3 fs as ext4 fs (without having to upgrade it).
  
• several other new features are introduced in ext4: multiblock allocation, delayed allocation, journal checksum, fast fsck, etc.
  
• in ext4, you also have the option of turning the journaling feature "off".

There are three types of journaling available in ext3/ext4 file systems:

• Journal - metadata and content are saved in the journal.
  
• Ordered - only metadata is saved in the journal. Metadata are journaled only after writing the content to disk. This is the default.
  
• Writeback - only metadata is saved in the journal. Metadata might be journaled either before or after the content is written to the disk.

An inode is a data structure on a filesystem on Linux and other Unix-like operating systems that stores all the information about a file except its name and its actual data. A data structure is a way of storing data so that it can be used efficiently.

A Unix file is "stored" in two different parts of the disk - the data blocks and the inodes. (I won't get into superblocks and other esoteric information.) The data blocks contain the "contents" of the file. The information about the file is stored elsewhere - in the inode.

Use the lvextend command for resize LVM partition.

• extending the size by 500MB:

lvextend -L +500M /dev/vgroup/lvolume

• extending all available free space:

lvextend -l +100%FREE /dev/vgroup/lvolume

and resize2fs or xfs_growfs to resize filesystem:

• for ext filesystems:

resize2fs /dev/vgroup/lvolume

• for xfs filesystem:

xfs_growfs mountpoint_for_/dev/vgroup/lvolume

1. Create partition

fdisk /dev/sdb

1. Create LVM

pvcreate /dev/sdb1
vgcreate vg0 /dev/sdb1
lvcreate --name datastore --size 50G vg0

1. Create filesystem

mkfs -t xfs /dev/mapper/vg0-datastore

The most significant advantage of executing the running process in the background is that you can do any other task simultaneously while other processes are running in the background. So, more processes can be completed in the background while you are working on different processes. It can be achieved by adding a special character '&' at the end of the command.

Is a process that has completed execution (via the exit system call) but still has an entry in the process table: it is a process in the "Terminated state".

kill -9 (SIGKILL) always works, provided you have the permission to kill the process. Basically either the process must be started by you and not be setuid or setgid, or you must be root. There is one exception: even root cannot send a fatal signal to PID 1 (the init process).

However kill -9 is not guaranteed to work immediately. All signals, including SIGKILL, are delivered asynchronously: the kernel may take its time to deliver them. Usually, delivering a signal takes at most a few microseconds, just the time it takes for the target to get a time slice. However, if the target has blocked the signal, the signal will be queued until the target unblocks it.

Normally, processes cannot block SIGKILL. But kernel code can, and processes execute kernel code when they call system calls.

A process blocked in a system call is in uninterruptible sleep. The ps or top command will (on most unices) show it in state D.

A classical case of long uninterruptible sleep is processes accessing files over NFS when the server is not responding; modern implementations tend not to impose uninterruptible sleep (e.g. under Linux, the intr mount option allows a signal to interrupt NFS file accesses).

You may sometimes see entries marked Z (or H under Linux) in the ps or top output. These are technically not processes, they are zombie processes, which are nothing more than an entry in the process table, kept around so that the parent process can be notified of the death of its child. They will go away when the parent process pays attention (or dies).

strace is a powerful command line tool for debugging and trouble shooting programs in Unix-like operating systems such as Linux. It captures and records all system calls made by a process and the signals received by the process.

The algorithms supported are MD5, Blowfish, SHA256 and SHA512.

Key-based authentication is a kind of authentication that may be used as an alternative to password authentication. Instead of requiring a user's password, it is possible to confirm the client's identity by using asymmetric cryptography algorithms, with public and private keys.

logrotate command is used to make automate rotation of log. It allows automatic rotation, compression, removal, and mailing of log files.

Most UNIX-like operating systems, including Linux and BSD, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis. These "ulimits" prevent single users from using too many system resources.

Hard limit is the maximum allowed to a user, set by the superuser or root. This value is set in the file /etc/security/limits.conf. The user can increase the soft limit on their own in times of needing more resources, but cannot set the soft limit higher than the hard limit.

• one of the main difference between cron and anacron jobs is that cron works on the system that are running continuously that means it is designed for the system that is running24*7. While anacron is used for the systems that are not running continuously.
  
• other difference between the two is cron jobs can run every minute, but anacron jobs can be run only once a day.
  
• any normal user can do the scheduling of cron jobs, but the scheduling of anacron jobs can be done by the superuser only.
  
• cron should be used when you need to execute the job at a specific time as per the given time in cron, but anacron should be used in when there is no any restriction for the timing and can be executed at any time.

For example:

for i in $(seq 1 100000) ; do
  rand=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)
  echo "$rand" >> /path/to/file
done

For example (with visudo command):

user1 ALL=(user2) NOPASSWD: /opt/scripts/bin/generate.sh

The command paths must be absolute! Then call sudo -u user2 /opt/scripts/bin/generate.sh from a user1 shell.

In a bash script, you have several ways to check if the running user is root.

As a warning, do not check if a user is root by using the root username. Nothing guarantees that the user with ID 0 is called root. It's a very strong convention that is broadly followed but anybody could rename the superuser another name.

I think the best way when using bash is to use $EUID because $UID could be changed and not reflect the real user running the script.

if (( $EUID != 0 )); then
  echo "Please run as root"
  exit
fi

For example:

# With -j8 - performance optimization
git clone --recurse-submodules -j8 git://github.com/foo/bar.git

# For already cloned repos or older Git versions
git clone git://github.com/foo/bar.git
cd bar
git submodule update --init --recursive

The command you want is named tee:

foo | tee output.file

For example, if you only care about stdout:

ls -a | tee output.file

If you want to include stderr, do:

program [arguments...] 2>&1 | tee outfile

2>&1 redirects channel 2 (stderr/standard error) into channel 1 (stdout/standard output), such that both is written as stdout. It is also directed to the given output file as of the tee command.

Furthermore, if you want to append to the log file, use tee -a as:

program [arguments...] 2>&1 | tee -a outfile

You should use #!/usr/bin/env bash for portability: different *nixes put bash in different places, and using /usr/bin/env is a workaround to run the first bash found on the PATH.

ssh-keygen -lf ~/.ssh/id_rsa.pub

A "root" authority is a certificate issuer that parties choose to trust (explicitly). It is usually self-signed (self-issued) and very highly protected. An intermediate authority is a certificate issuer that has itself been issued by a root or another higher level intermediate authority.

Self-signed root certificates need not/should not be included in web server configuration. They serve no purpose (clients will always ignore them) and they incur a slight performance (latency) penalty because they increase the size of the SSL handshake.

If the client does not have the root in their trust store, then it won't trust the web site, and there is no way to work around that problem. Having the web server send the root certificate will not help - the root certificate has to come from a trusted 3rd party (in most cases the browser vendor).

Solution 1:

systemctl reload postgresql

Solution 2:

su - postgres
/usr/bin/pg_ctl reload

Solution 3:

SELECT pg_reload_conf();

Will recreate any missing configuration files, e.g. /etc/mysql/my.cnf in your case:

dpkg -i --force-confmiss mysql-common.deb

The best way is: exec $SHELL -l

kill -9 $$
unset HISTFILE && exit

For example use fgrep:

fgrep * -R "string"

or:

grep -insr "pattern" *

• -i ignore case distinctions in both the PATTERN and the input files
• -n prefix each line of output with the 1-based line number within its input file
• -s suppress error messages about nonexistent or unreadable files.
• -r read all files under each directory, recursively.

You can do this with ldd command:

ldd /bin/ls

Handshaking begins when one device sends a message to another device indicating that it wants to establish a communications channel. The two devices then send several messages back and forth that enable them to agree on a communications protocol.

A three-way handshake is a method used in a TCP/IP network to create a connection between a local host/client and server. It is a three-step method that requires both the client and server to exchange SYN and ACK (SYN, SYN-ACK, ACK) packets before actual data communication begins.

UDP is faster than TCP, and the simple reason is because its nonexistent acknowledge packet (ACK) that permits a continuous packet stream, instead of TCP that acknowledges a set of packets, calculated by using the TCP window size and round-trip time (RTT).

It enables private IP networks that use unregistered IP addresses to connect to the Internet. NAT operates on a router, usually connecting two networks together, and translates the private (not globally unique) addresses in the internal network into legal addresses, before packets are forwarded to another network.

Workstations or other computers requiring special access outside the network can be assigned specific external IPs using NAT, allowing them to communicate with computers and applications that require a unique public IP address. NAT is also a very important aspect of firewall security.

This protocol operates at layer 2 of the OSI model with the purpose of preventing loops on the network. Without STP, a redundant switch deployment would create broadcast storms that cripple even the most robust networks. There are several iterations based on the original IEEE 802.1D standard; each operates slightly different than the others while largely accomplishing the same loop-free goal.

Use the:

• lsof -i
• ss -l
• netstat -atn - for tcp
• netstat -aun - for udp
• netstat -tulapn

For example:

telnet example.com 80
Trying 192.168.252.10...
Connected to example.com.
Escape character is '^]'.
GET /questions HTTP/1.0
Host: example.com

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
...

For example:

/sbin/iptables -A INPUT -p tcp -s XXX.XXX.XXX.XXX -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -d  XXX.XXX.XXX.XXX -j ACCEPT

The best way to do this is to define a table and create a rule to block the hosts, in pf.conf:

table <badhosts> persist
block on fxp0 from <badhosts> to any

And then dynamically add/delete IP addresses from it:

pfctl -t badhosts -T add 1.2.3.4
pfctl -t badhosts -T delete 1.2.3.4

For example:

sub vcl_recv {
   # dont cache foo.com or bar.com - optional www
   if (req.host ~ "(www)?(foo|bar).com") {
     return(pass);
   }
}

or:

sub vcl_recv {
  # dont cache foo.com or bar.com - optional www
   if (req.http.host ~ "(www\.)?(foo|bar)\.com") {
     return(pass);
   }
  # cache foobar.com - optional www
   if (req.http.host ~ "(www\.)?foobar\.com") {
     return(hash);
   }
}

tail -n 100 -f /path/to/logfile | grep "HTTP/[1-2].[0-1]\" [5]"

CORS allows the Same Origin Policy to be relaxed for a domain.

e.g. normally if the user logs into both example.com and example.org, the Same Origin Policy prevents example.com from making an AJAX request to example.org/current_user/full_user_details and gaining access to the response.

This is the default policy of the web and prevents the user's data from being leaked when logged into multiple sites at the same time.

Now with CORS, example.org could set a policy to say it will allow the origin https://example.com to read responses made by AJAX. This would be done if both example.com and example.org are ran by the same company and data sharing between the origins is to be allowed in the user's browser. It only affects the client-side of things, not the server-side.

CSPs on the other hand set a policy of what content can run on the current site. For example, if JavaScript can be executed inline, or which domains .js files can be loaded from. This can be beneficial to act as another line of defence against XSS attacks, where the attacker will try and inject script into the HTML page. Normally output would be encoded, however say the developer had forgotten only on one output field. Because the policy is preventing in-line script from executing, the attack is thwarted.

There might be four types of responses:

• Open port - few ports in the case of the firewall
• Closed port - most ports are closed because of the firewall
• Filtered - Nmap is not sure whether the port is open or not
• Unfiltered - Nmap can access the port but is still confused about the open status of the port

The most popular DevOps tools are mentioned below:

• Git : Version Control System tool
• Jenkins : Continuous Integration tool
• Selenium : Continuous Testing tool
• Puppet, Chef, Ansible : Configuration Management and Deployment tools
• Nagios : Continuous Monitoring tool
• Docker : Containerization tool

The most popular DevOps tools are mentioned below:

• Developers develop the code and this source code is managed by Version Control System tools like Git etc.
• Developers send this code to the Git repository and any changes made in the code is committed to this Repository.
• Jenkins pulls this code from the repository using the Git plugin and build it using tools like Ant or Maven.
• Configuration management tools like puppet deploys & provisions testing environment and then Jenkins releases this code on the test environment on which testing is done using tools like selenium.
• Once the code is tested, Jenkins send it for deployment on the production server (even production server is provisioned & maintained by tools like puppet).
• After deployment It is continuously monitored by tools like Nagios.
• Docker containers provides testing environment to test the build features.

Playbooks are Ansible’s configuration, deployment, and orchestration language.

They can describe a policy you want your remote systems to enforce, or a set of steps in a general IT process. Playbooks are designed to be human-readable and are developed in a basic text language.

At a basic level, playbooks can be used to manage configurations of and deployments to remote machines.

The NRPE addon is designed to allow you to execute Nagios plugins on remote Linux/Unix machines. The main reason for doing this is to allow Nagios to monitor "local" resources (like CPU load, memory usage, etc.) on remote machines.

Since these public resources are not usually exposed to external machines, an agent like NRPE must be installed on the remote Linux/Unix machines.

The major difference between Active and Passive checks is that Active checks are initiated and performed by Nagios, while passive checks are performed by external applications.

Passive checks are useful for monitoring services that are:

• asynchronous in nature and cannot be monitored effectively by polling their status on a regularly scheduled basis.
• located behind a firewall and cannot be checked actively from the monitoring host.

The main features of Actives checks are as follows:

• active checks are initiated by the Nagios process.
• active checks are run on a regularly scheduled basis.

Cross Site Scripting is a JavaScript vulnerability in the web applications. The easiest way to explain this is a case when a user enters a script in the client side input fields and that input gets processed without getting validated. This leads to untrusted data getting saved and executed on the client side.

Countermeasures of XSS are input validation, implementing a CSP (Content security policy) etc.

HIDS is host intrusion detection system and NIDS is network intrusion detection system. Both the systems work on the similar lines. It’s just that the placement in different. HIDS is placed on each host whereas NIDS is placed in the network. For an enterprise, NIDS is preferred as HIDS is difficult to manage, plus it consumes processing power of the host as well.

Abiding by a set of standards set by a government/Independent party/organisation. E.g. An industry which stores, processes or transmits Payment related information needs to be complied with PCI DSS (Payment card Industry Data Security Standard). Other compliance examples can be an organisation complying with its own policies.

WAF stands for web application firewall. It is used to protect the application by filtering legitimate traffic from malicious traffic. WAF can be either a box type or cloud based.

?Senior Sysadmin

The specific answer depends on the situation/person.

The specific answer depends on the situation/person.

• Microkernel: This type of kernel only manages CPU, memory, and IPC. This kind of kernel provides portability, small memory footprint and also security.
  
• Monolithic Kernel: Linux is a monolithic kernel. So, this type of kernel provides file management, system server calls, also manages CPU, IPC as well as device drivers. It provides easier access to the process to communicate and as there is not any queue for processor time, so processes react faster.
  
• Hybrid Kernel: In this type of kernels, programmers can select what they want to run in user mode and what in supervisor mode. So, this kernel provides more flexibility than any other kernel but it can have some latency problems.

Environment variable LD_LIBRARY_PATH is a colon-separated set of directories where libraries should be searched for first, before the standard set of directories; this is useful when debugging a new library or using a nonstandard library for special purposes.

pwconv command is used for giving shadow passwords. Shadow passwords are given for better system security. The pwconv command creates the file /etc/shadow and changes all passwords to 'x' in the /etc/passwd file.

• Username: First field is the username that contains the username which is 1 to 32 length characters.
  
• Password: This field does not show the actual password as the password is encrypted. Here, x character shows that password is encrypted that is located in /etc/shadow file.
  
• User ID (UID): All the users created in Linux is given a user ID whenever the user is created. UID 0 is fixed and reserved for the root user.
  
• Group ID (GID): This field specifies the name of the group to which the user belongs. The group information is also stored in a file /etc/group.
  
• User ID Info: Here you can add comments and you can add any extra information related to the users like full name, contact number, etc.
  
• Home directory: This field provides the path where the user is directed after the login. For example, /home/smith.
  
• Command/shell: This field provides the path of a command/shell and denotes that user has access to this shell i.e. /bin/bash.

In BSD the primary start-up configuration file is /etc/defaults/rc.conf. System startup scripts such as /etc/rc and /etc/rc.d just include this file.

If you want to add other programs to system startup you need to change /etc/rc.conf file instead of /etc/defaults/rc.conf.

Sar collect, report or save system activity information. The default version of the sar command (CPU utilization report) might be one of the first facilities the user runs to begin system activity investigation, because it monitors major system resources. If CPU utilization is near 100 percent (user + nice + system), the workload sampled is CPU-bound.

By default log files of Sar command is located at /var/log/sa/sadd file, where the dd parameter indicates the current day.

Run the command: echo "---" >/sys/class/scsi_host/hostX/scan.

There are some system calls used in Linux for process management. These are as follows:

• Fork(): It is used to create a new process
  
• Exec(): It is used to execute a new process
  
• Wait(): It is used to make the process to wait
  
• Exit(): It is used to exit or terminate the process
  
• Getpid(): It is used to find the unique process ID
  
• Getppid(): It is used to check the parent process ID
  
• Nice(): It is used to bias the currently running process property

Interrupts means the processor is transferred temporarily to another program or function. When that program is completed, the processor will be given back to that program to complete the task.

Interrupt handler is the function that the kernel runs for a specific interrupt. It is also called Interrupt Service Routine. Interrupts handlers are the function that matches a particular prototype and enables the kernel to pass the handler information accurately.

For example:

# cat  >filename ... - overwrite file
# cat >>filename ... - append to file
cat > filename << __EOF__
data data data
__EOF__

Within a POSIX system, a file has the following attributes which may be retrieved by the stat system call:

• Device ID (this identifies the device containing the file; that is, the scope of uniqueness of the serial number). File serial numbers
• The file mode which determines the file type and how the file's owner, its group, and others can access the file
• A link count telling how many hard links point to the inode
• The User ID of the file's owner
• The Group ID of the file
• The device ID of the file if it is a device file.
• The size of the file in bytes
• Timestamps telling when the inode itself was last modified (ctime, inode change time), the file content last modified (mtime, modification time), and last accessed (atime, access time)
• The preferred I/O block size
• The number of blocks allocated to this file

It provides a layer between applications and actual authentication mechanism. It is a library of loadable modules which are called by the application for authentication. It also allows the administrator to control when a user can log in. All PAM applications are configured in the directory /etc/pam.d or in a file /etc/pam.conf. PAM is controlled using the configuration file or the configuration directory.

For example:

while inotifywait -e close_write filename ; do

  echo "changed" >> /var/log/changed

done

A superblock is a record of the characteristics of a filesystem, including its size, the block size, the empty and the filled blocks and their respective counts, the size and location of the inode tables, the disk block map and usage information, and the size of the block groups.

An "s" in the first position means that the SETUID (or SUID) bit was set (the GUID bit is the same thing, at the group level). Linux runs an executable file with the SETUID bit set with the User ID (that is, the privileges!) of the owner of that file, not the one of the user who launched it.

In Unix and Unix-like operating systems, kill is a command used to send a signal to a process. By default, the message sent is the termination signal, which requests that the process exit. But kill is something of a misnomer; the signal sent may have nothing to do with process killing.

• Zombie processes cannot be killed since they are already dead and waiting for their parent processes to reap them.
  
• Processes that are in the blocked state will not die until they wake up again.
  
• The init process is special: It does not get signals that it does not want to handle, and thus it can ignore SIGKILL. An exception from this exception is while init is ptraced on Linux.
  
• An uninterruptibly sleeping process may not terminate (and free its resources) even when sent SIGKILL. This is one of the few cases in which a UNIX system may have to be rebooted to solve a temporary software problem.

Speaking of killing processes never use kill -9/SIGKILL unless absolutely mandatory. This kill can cause problems because of its brute force.

Always try to use the following simple procedure:

• first, send SIGTERM (kill -15) signal first which tells the process to shutdown and is generally accepted as the signal to use when shutting down cleanly (but remember that this signal can be ignored).
• next try to send SIGHUP (kill -1) signal which is commonly used to tell a process to shutdown and restart, this signal can also be caught and ignored by a process.

The far majority of the time, this is all you need - and is much cleaner.

The command setfacl refers to Set File Access Control Lists and getfacl refers to Get File Access Control List. Each file and directory in a Linux filesystem is created with a specific set of file permissions for its access. In order to know the access permissions of a file or directory we use getfacl.

In Unix and related computer operating systems, a file descriptor (FD, less frequently fildes) is an abstract indicator (handle) used to access a file or other input/output resource, such as a pipe or network socket. File descriptors form part of the POSIX application programming interface.

The process table entry (aka process control block) contains a table, the file descriptor table that gives the mapping between the descriptor the process uses to refer to a file connection and the data structure inside the kernel that represents the actual file connection.

When /sbin/nologin is set as the shell, if user with that shell logs in, they'll get a polite message saying 'This account is currently not available.'

/bin/false is just a binary that immediately exits, returning false, when it's called, so when someone who has false as shell logs in, they're immediately logged out when false exits. Setting the shell to /bin/true has the same effect of not allowing someone to log in but false is probably used as a convention over true since it's much better at conveying the concept that person doesn't have a shell.

nologin is the more user-friendly option, with a customizable message given to the user trying to log in, so you would theoretically want to use that; but both nologin and false will have the same end result of someone not having a shell and not being able to ssh in.

The FreeBSD kernel will only allow a certain number of processes to exist at one time. The number is based on the kern.maxusers variable.

kern.maxusers also affects various other in-kernel limits, such as network buffers. If the machine is heavily loaded, increase kern.maxusers. This will increase these other system limits in addition to the maximum number of processes.

To adjust the kern.maxusers value, see the File/Process Limits section of the Handbook. While that section refers to open files, the same limits apply to processes.

If the machine is lightly loaded but running a very large number of processes, adjust the kern.maxproc tunable by defining it in /boot/loader.conf.

For example:

while IFS='' read -r line || [[ -n "$line" ]] ; do
  echo "Text read from file: $line"
done < "/path/to/filename"

Explanation:

• IFS='' (or IFS=) prevents leading/trailing whitespace from being trimmed.
• -r prevents backslash escapes from being interpreted.
• || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).

Debian GNU/Linux:

update-initramfs -m -k all
update-grub

CentOS Linux:

dracut -f
grub2-mkconfig -o /boot/grub/grub.cfg

An OS is a very busy thing, particularly so when you have it doing something (and even when you aren't). And when we are looking at an active enterprise environment, something is always going on.

Most of this activity is "bursty", meaning processes are typically quiescent with short periods of intense activity. This is certainly true of any type of network-based activity (e.g. processing PHP requests), but also applies to OS maintenance (e.g. file system maintenance, page reclamation, disk I/O requests).

If you take a situation where you have a lot of such bursty processes, you get a very irregular and spiky CPU usage plot.

As "500 - Internal Server Error" says, the high number of context switches are going to make the situation even worse.

strace -p <PID> - to attach a process to strace.

strace -e trace=read,write -p <PID> - by this you can also trace a process/program for an event, like read and write (in this example). So here it will print all such events that include read and write system calls by the process.

Other such examples

• -e trace= network - trace all the network related system calls.
• -e trace=signal - trace all signal related system calls.
• -e trace=ipc - trace all IPC related system calls.
• -e trace=desc - trace all file descriptor related system calls.
• -e trace=memory - trace all memory mapping related system calls.

Solution 1 - with extglob:

shopt -s extglob
rm !(textfile.txt|backup.tar.gz|script.php|database.sql|info.txt)

Solution 2 - with find:

find . -type f -not -name '*txt' -print0 | xargs -0 rm --

You can use * (wildcards) outside a case statement, too, if you use double brackets:

string='some text'
if [[ $string = *"My long"* ]] ; then
  true
fi

Just add them in one line command 2>> error 1>> output.

However, note that >> is for appending if the file already has data. Whereas, > will overwrite any existing data in the file.

So, command 2> error 1> output if you do not want to append.

Just for completion's sake, you can write 1> as just > since the default file descriptor is the output. so 1> and > is the same thing.

So, command 2> error 1> output becomes, command 2> error > output.

Warning: this will overwrite the original file:

sed -i "s/^[ \t]*//" filename

or:

sed 's/^[ \t]+//g' < input > output

Force login with a password:

ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@remote_host

Force login using the key:

ssh -o PreferredAuthentications=publickey -o PubkeyAuthentication=yes -i id_rsa user@remote_host

Fixed the issue by reducing max_files_per_process e.g. to 200 from default 1000. This parameter is in postgresql.conf file and this sets the maximum number of simultaneously open files allowed to each server subprocess.

Usually people start to edit /etc/security/limits.conf file, but forget that this file only apply to the actively logged in users through the pam system.

Solution 1:

Check for files on located under mount points. Frequently if you mount a directory (say a sambafs) onto a filesystem that already had a file or directories under it, you lose the ability to see those files, but they're still consuming space on the underlying disk.

I've had file copies while in single user mode dump files into directories that I couldn't see except in single usermode (due to other directory systems being mounted on top of them).

Solution 2:

On the other hand df -h and du -sh could mismatched by about 50% of the hard disk size. This was caused by e.g. apache (httpd) keeping large log files in memory which had been deleted from disk.

This was tracked down by running lsof | grep "/var" | grep deleted where /var was the partition I needed to clean up.

The output showed lines like this:

httpd     32617    nobody  106w      REG        9,4 1835222944     688166 /var/log/apache/awstats_log (deleted)

The situation was then resolved by restarting apache (service httpd restart) and cleared of disk space, by allowing the locks on deleted files to be cleared.

Hashing: Finally, hashing is a form of cryptographic security which differs from encryption. Whereas encryption is a two step process used to first encrypt and then decrypt a message, hashing condenses a message into an irreversible fixed-length value, or hash.

auditd is the correct tool for the job here:

1. Add these 2 lines to /etc/audit/audit.rules:

-a exit,always -F arch=b64 -F euid=0 -S execve
-a exit,always -F arch=b32 -F euid=0 -S execve

These will track all commands run by root (euid=0). Why two rules? The execve syscall must be tracked in both 32 and 64 bit code.

1. To get rid of auid=4294967295 messages in logs, add audit=1 to the kernel's cmdline (by editing /etc/default/grub)

2. Place the line

session  required                pam_loginuid.so

in all PAM config files that are relevant to login (/etc/pam.d/{login,kdm,sshd}), but not in the files that are relevant to su or sudo. This will allow auditd to get the calling user's uid correctly when calling sudo or su.

Restart your system now.

Let's login and run some commands:

$ id -u
1000
$ sudo ls /
bin  boot  data  dev  etc  home  initrd.img  initrd.img.old  lib  lib32  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  scratch  seLinux  srv  sys  tmp  usr  var  vmlinuz  vmlinuz.old
$ sudo su -
# ls /etc
[...]

Now read /var/log/audit/auditd.log for show what has been logged in.

Try using ionice:

ionice -c3 dd if=/dev/zero of=z

This start the dd process with the "idle" IO priority: it only gets disk time when no other process is using disk IO for a certain amount of time.

Of course this can still flood the buffer cache and cause freezes while the system flushes out the cache to disk. There are tunables under /proc/sys/vm/ to influence this, particularly the dirty_* entries.

• nice/renice

nice is a great tool for 'one off' tweaks to a system:

nice COMMAND

• cpulimit

cpulimit if you need to run a CPU intensive job and having free CPU time is essential for the responsiveness of a system:

cpulimit -l 50 COMMAND

• cgroups

cgroups apply limits to a set of processes, rather than to just one:

cgcreate -g cpu:/cpulimited
cgset -r cpu.shares=512 cpulimited
cgexec -g cpu:cpulimited COMMAND_1
cgexec -g cpu:cpulimited COMMAND_2
cgexec -g cpu:cpulimited COMMAND_3

# -t - filesystem type
# -o - mount options
mount -t tmpfs tmpfs /mnt -o size=64M

fuser -k filename

# 1:
cp /bin/ls chmod.01
cp /bin/chmod chmod.01
./chmod.01 700 file

# 2:
/bin/busybox chmod 0700 /bin/chmod

# 3:
setfacl --set u::rwx,g::---,o::--- /bin/chmod

• grub> - this is the mode to which it passes if you find everything you need to run the system in addition to the configuration file. With this mode, we have access to most (if not all) modules and commands. This mode can be called from the menu by pressing the 'c' key
• grub-rescue> - this is the mode to which it passes if it is impossible to find its own directory (especially the directory with modules and additional commands, e.g. directory /boot/grub/i386-pc), if its contents are damaged or if no normal module is found, contains only basic commands

(openssl rsa -noout -modulus -in private.key | openssl md5 ; openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq

1. Add an entry of user details in /etc/passwd

# username:password:UID:GID:Comments:Home_Directory:Login Shell
user:x:501:501:test user:/home/user:/bin/bash

1. You will have to create a group with same name in /etc/group

user:x:501:

1. Assign a password to the user

passwd user

Ideally, I need an app that will attach to a process and log periodic snapshots of: memory usage number of threads CPU usage.

1. You can use top in batch mode. It runs in the batch mode either until it is killed or until N iterations is done:

top -b -p `pidof a.out`

or

top -b -p `pidof a.out` -n 100

1. You can use ps (for instance in a shell script):

ps --format pid,pcpu,cputime,etime,size,vsz,cmd -p `pidof a.out`

I need some means of recording the performance of an application on a Linux machine.

1. To record performance data:

perf record -p `pidof a.out`

or to record for 10 secs:

perf record -p `pidof a.out` sleep 10

or to record with call graph ():

perf record -g -p `pidof a.out`

1. To analyze the recorded data

perf report --stdio
perf report --stdio --sort=dso -g none
perf report --stdio -g none
perf report --stdio -g

This is an example of profiling a test program

1. I run my test program (c++):

./my_test 100000000

1. Then I record performance data of a running process:

perf record -g  -p `pidof my_test` -o ./my_test.perf.data sleep 30

1. Then I analyze load per module:

perf report --stdio -g none --sort comm,dso -i ./my_test.perf.data

# Overhead  Command                 Shared Object
# ........  .......  ............................
#
    70.06%  my_test  my_test
    28.33%  my_test  libtcmalloc_minimal.so.0.1.0
     1.61%  my_test  [kernel.kallsyms]

1. Then load per function is analyzed:

perf report --stdio -g none -i ./my_test.perf.data | c++filt

# Overhead  Command                 Shared Object                       Symbol
# ........  .......  ............................  ...........................
#
    29.30%  my_test  my_test                       [.] f2(long)
    29.14%  my_test  my_test                       [.] f1(long)
    15.17%  my_test  libtcmalloc_minimal.so.0.1.0  [.] operator new(unsigned long)
    13.16%  my_test  libtcmalloc_minimal.so.0.1.0  [.] operator delete(void*)
     9.44%  my_test  my_test                       [.] process_request(long)
     1.01%  my_test  my_test                       [.] operator delete(void*)@plt
     0.97%  my_test  my_test                       [.] operator new(unsigned long)@plt
     0.20%  my_test  my_test                       [.] main
     0.19%  my_test  [kernel.kallsyms]             [k] apic_timer_interrupt
     0.16%  my_test  [kernel.kallsyms]             [k] _spin_lock
     0.13%  my_test  [kernel.kallsyms]             [k] native_write_msr_safe

  ...

1. Then call chains are analyzed:

perf report --stdio -g graph -i ./my_test.perf.data | c++filt

# Overhead  Command                 Shared Object                       Symbol
# ........  .......  ............................  ...........................
#
    29.30%  my_test  my_test                       [.] f2(long)
            |
            --- f2(long)
               |
                --29.01%-- process_request(long)
                          main
                          __libc_start_main

    29.14%  my_test  my_test                       [.] f1(long)
            |
            --- f1(long)
               |
               |--15.05%-- process_request(long)
               |          main
               |          __libc_start_main
               |
                --13.79%-- f2(long)
                          process_request(long)
                          main
                          __libc_start_main

  ...

So at this point you know where your program spends time.

Also the simple way to do app profile is to use the pstack utility or lsstack.

Other tool is Valgrind. So this is what I recommend. Run program first:

valgrind --tool=callgrind --dump-instr=yes -v --instr-atstart=no ./binary > tmp

Now when it works and we want to start profiling we should run in another window:

callgrind_control -i on

This turns profiling on. To turn it off and stop whole task we might use:

callgrind_control -k

Now we have some files named callgrind.out.* in current directory. To see profiling results use:

kcachegrind callgrind.out.*

I recommend in next window to click on "Self" column header, otherwise it shows that "main()" is most time consuming task.

They're effective but not optimally portable:

• rm -- -fr
• perl -le 'unlink("-fr");'

People who go on about shell command line quoting and character escaping are almost as dangerous as those who simply don't even recognize why a file name like that poses any problem at all.

The most portable solution:

rm ./-fr

To list any process listening to the port 8080:

lsof -i:8080

To kill any process listening to the port 8080:

kill $(lsof -t -i:8080)

or more violently:

kill -9 $(lsof -t -i:8080)

With OpenSSL:

# Testing connection to remote host (with SNI support)
echo | openssl s_client -showcerts -servername google.com -connect google.com:443
# Testing connection to remote host (without SNI support)
echo | openssl s_client -connect google.com:443 -showcerts

With GnuTLS:

# Testing connection to remote host (with SNI support)
gnutls-cli -p 443 google.com
# Testing connection to remote host (without SNI support)
gnutls-cli --disable-sni -p 443 google.com

The server sends the following in its response header to set a cookie field:

Set-Cookie:name=value

If there is a cookie set, then the browser sends the following in its request header:

Cookie:name=value

The best benchmark is always "the application(s) that you normally use". The load on a NFS system when you have 20 people simultaneously compiling a Linux kernel is comletely different from a bunch of people logging in at the same time or the accounts uses as "home directories for the local web-server".

But we have some good tools for testing this.

• boonie - a classical performances evaluation tool tests. The main program tests database type access to a single file (or a set of files if you wish to test more than 1G of storage), and it tests creation, reading, and deleting of small files which can simulate the usage of programs such as Squid, INN, or Maildir format email.
• DBench - was written to allow independent developers to debug and test SAMBA. It is heavily inspired of the original SAMBA tool.
• IOZone - performance tests suite. POSIX and 64 bits compliant. This tests is the file system test from the L.S.E. Main features: POSIX async I/O, Mmap() file I/O, Normal file I/O Single stream measurement, Multiple stream measurement, Distributed file server measurements (Cluster) POSIX pthreads, Multi-process measurement Selectable measurements with fsync, O_SYNC Latency plots.

With ssh:

ssh user1@remote1 'ssh user2@remote2 "cat file"' > file

With tar (with compression):

ssh user1@remote1 'ssh user2@remote2 "cd path2; tar cj file"' | tar xj

With ssh and port forwarding tunnel:

# First, open the tunnel
ssh -L 1234:remote2:22 -p 45678 user1@remote1
# Then, use the tunnel to copy the file directly from remote2
scp -P 1234 user2@localhost:file .

• webpage optimization
• cached web pages
• quality web hosting
• compressed text files
• apache/nginx tuning

HTTP/2 supports queries multiplexing, headers compression, priority and more intelligent packet streaming management. This results in reduced latency and accelerates content download on modern web pages.

Key differences with HTTP/1.1:

• it is binary, instead of textual
• fully multiplexed, instead of ordered and blocking
• can therefore use one connection for parallelism
• uses header compression to reduce overhead
• allows servers to “push” responses proactively into client caches

When sent by the origin server:

max-age=0 simply tells caches (and user agents) the response is stale from the get-go and so they SHOULD revalidate the response (e.g. with the If-Not-Modified header) before using a cached copy, whereas, no-cache tells them they MUST revalidate before using a cached copy.

In other words, caches may sometimes choose to use a stale response (although I believe they have to then add a Warning header), but no-cache says they're not allowed to use a stale response no matter what. Maybe you'd want the SHOULD-revalidate behavior when baseball stats are generated in a page, but you'd want the MUST-revalidate behavior when you've generated the response to an e-commerce purchase.

When sent by the user agent:

If a user agent sends a request with Cache-Control: max-age=0 (aka. "end-to-end revalidation"), then each cache along the way will revalidate its cache entry (e.g. with the If-Not-Modified header) all the way to the origin server. If the reply is then 304 (Not Modified), the cached entity can be used.

On the other hand, sending a request with Cache-Control: no-cache (aka. "end-to-end reload") doesn't revalidate and the server MUST NOT use a cached copy when responding.

By responding with Access-Control-Allow-Origin: *, the requested resource allows sharing with every origin. This basically means that any site can send an XHR request to your site and access the server’s response which would not be the case if you hadn’t implemented this CORS response.

So any site can make a request to your site on behalf of their visitors and process its response. If you have something implemented like an authentication or authorization scheme that is based on something that is automatically provided by the browser (cookies, cookie-based sessions, etc.), the requests triggered by the third party sites will use them too.

### TCP -> TCP
nc -l -p 2000 -c "nc [ip|hostname] 3000"

### TCP -> UDP
nc -l -p 2000 -c "nc -u [ip|hostname] 3000"

### UDP -> UDP
nc -l -u -p 2000 -c "nc -u [ip|hostname] 3000"

### UDP -> TCP
nc -l -u -p 2000 -c "nc [ip|hostname] 3000"

1. Use Decoy addresses

# Generates a random number of decoys.
nmap -D RND:10 [target]
# Manually specify the IP addresses of the decoys.
nmap -D decoy1,decoy2,decoy3

In this type of scan you can instruct Nmap to spoof packets from other hosts.In the firewall logs it will be not only our IP address but also and the IP addresses of the decoys so it will be much harder to determine from which system the scan started.

1. Source port number specification

nmap --source-port 53 [target]

A common error that many administrators are doing when configuring firewalls is to set up a rule to allow all incoming traffic that comes from a specific port number.The --source-port option of Nmap can be used to exploit this misconfiguration.Common ports that you can use for this type of scan are: 20, 53 and 67.

1. Append Random Data

nmap --data-length 25 [target]

Many firewalls are inspecting packets by looking at their size in order to identify a potential port scan.This is because many scanners are sending packets that have specific size.In order to avoid that kind of detection you can use the command --data-length to add additional data and to send packets with different size than the default.

1. TCP ACK Scan

nmap -sA [target]

It is always good to send the ACK packets rather than the SYN packets because if there is any active firewall working on the remote computer then because of the ACK packets the firewall cannot create the log, since firewalls treat ACK packet as the response of the SYN packet.

tcpdump is a most powerful and widely used command-line packets sniffer or package analyzer tool which is used to capture or filter TCP/IP packets that received or transferred over a network on a specific interface.

tcpdump puts your network card into promiscuous mode, which basically tells it to accept every packet it receives. It allows the user to see all traffic being passed over the network. Wireshark uses pcap to capture packets.

If you want to view only packets that come to your interface you should:

• -Q in - for Linux tcpdump version
• -D in - for BSD tcpdump version

Both params set send/receive direction direction for which packets should be captured.

tcpdump -nei eth0 -Q in host 192.168.252.125 and port 8080

Flapping occurs when a service or host changes state too frequently, this causes lot of problem and recovery notifications.

Once you have defined Flapping, explain how Nagios detects Flapping. Whenever Nagios checks the status of a host or service, it will check to see if it has started or stopped flapping.

Nagios follows the below given procedure to do that:

• storing the results of the last 21 checks of the host or service analyzing the historical check results and determine where state changes/transitions occur
• using the state transitions to determine a percent state change value (a measure of change) for the host or service
• comparing the percent state change value against low and high flapping thresholds

Below are the advantages of containerization over virtualization:

• containers provide real-time provisioning and scalability but VMs provide slow provisioning
• containers are lightweight when compared to VMs
• VMs have limited performance when compared to containers
• containers have better resource utilization compared to VMs

Cross Site Request Forgery is a web application vulnerability in which the server does not check whether the request came from a trusted client or not. The request is just processed directly. It can be further followed by the ways to detect this, examples and countermeasures.

As security policy defines the security objectives and the security framework of an organisation. A process is a detailed step by step how to document that specifies the exact action which will be necessary to implement important security mechanism. Guidelines are recommendations which can be customised and used in the creation of procedures.

When the device generated an alert for an intrusion which has actually not happened: this is false positive and if the device has not generated any alert and the intrusion has actually happened, this is the case of a false negative.

Web server hardening is filtering of unnecessary services running on various ports and removal of default test scripts from the servers. Although web server hardening is a lot more than this and usually organisations have a customised checklist for hardening the servers. Any server getting created has to be hardened and hardening has to be re-confirmed on a yearly basis. Even the hardening checklist has to be reviewed on a yearly basis for new add-ons.

Example:

• if machine is a new install, protect it from hostile network traffic, until the operating system is installed and hardened
• create a separate partition with the nodev, nosuid, and noexec options set for /tmp
• create separate partitions for /var, /var/log, /var/log/audit, and /home
• enable randomized virtual memory region placement
• remove legacy services (e.g., telnet-server; rsh, rlogin, rcp; ypserv, ypbind; tftp, tftp-server; talk, talk-server).
• limit connections to services running on the host to authorized users of the service via firewalls and other access control technologies
• disable source routed packet acceptance
• enable TCP/SYN cookies
• disable SSH root login
• install and configure AIDE
• install and configure OSsec HIDS
• configure SELinux
• all administrator or root access must be logged
• integrity checking of system accounts, group memberships, and their associated privileges should be enabled and tested
• set password creation requirements

Secret Knowledge

?Guru Sysadmin

mktemp randomizes the name. It is very important from the security point of view.

Just imagine that you do something like:

echo "random_string" > /tmp/temp-file

in your root-running script. And someone (who has read your script) does

ln -s /etc/passwd /tmp/temp-file

The mktemp command could help you in this situation:

TEMP=$(mktemp /tmp/temp-file.XXXXXXXX)
echo "random_string" > ${TEMP}

Now this ln /etc/passwd attack will not work.

For example:

#!/usr/bin/bash

for iface in $(ls /sys/class/net/ | grep -v lo) ; do

  if [[ $(cat /sys/class/net/$iface/carrier) = 1 ]] ; then state=1 ; fi

done

if [[ $state -ne 0 ]] ; then echo "not connection" > /dev/stderr ; exit ; fi

Every poll is recorded in the shared memory log as follows:

0 Backend_health - b0 Still healthy 4--X-S-RH 9 8 10 0.029291 0.030875 HTTP/1.1 200 Ok

The fields are:

• 0 - Constant
• Backend_health - Log record tag
• '-' client/backend indication (XXX: wrong! should be 'b')
• b0 - Name of backend (XXX: needs qualifier)
• two words indicating state:
  • "Still healthy"
  • "Still sick"
  • "Back healthy"
  • "Went sick"

Notice that the second word indicates present state, and the first word == "Still" indicates unchanged state.

• 4--X-S-RH - Flags indicating how the latest poll went
  • 4 - IPv4 connection established
  • 6 - IPv6 connection established
  • x - Request transmit failed
  • X - Request transmit succeeded
  • s - TCP socket shutdown failed
  • S - TCP socket shutdown succeeded
  • r - Read response failed
  • R - Read response succeeded
  • H - Happy with result
• 9 - Number of good polls in the last .window polls
• 8 - .threshold (see above)
• 10 - .window (see above)
• 0.029291 - Response time this poll or zero if it failed
• 0.030875 - Exponential average (r=4) of responsetime for good polls.
• HTTP/1.1 200 Ok - The HTTP response from the backend.

You just need to write a Nginx rewrite rule with HTTP status code 307 or 308:

location / {
    proxy_pass              http://localhost:80;
    client_max_body_size    10m;
}

location /api {
    # HTTP 307 only for POST method.
    if ($request_method = POST) {
        return 307 https://api.example.com?request_uri;
    }

    # You can keep this for non-POST requests.
    rewrite ^ https://api.example.com?request_uri permanent;

    client_max_body_size    10m;
}

HTTP Status code 307 or 308 should be used instead of 301 because it changes the request method from POST to GET.

# <proto - set protocol (tcp/udp)
# <host> - set remote host
# <port> - set destination port
timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?

cURL supports formatted output for the details of the request (see the cURL manpage for details, under -w| -write-out 'format'). For our purposes we’ll focus just on the timing details that are provided.

1. Create a new file, curl-format.txt, and paste in:

    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n

1. Make a request:

curl -w "@curl-format.txt" -o /dev/null -s "http://example.com/"

What this does:

• -w "@curl-format.txt" - tells cURL to use our format file
• -o /dev/null - redirects the output of the request to /dev/null
• -s - tells cURL not to show a progress meter http://example.com/ is the URL we are requesting. Use quotes particularly if your URL has "&" query string parameters

To match a list of domain and subdomain this regex make it ease to work with fonts:

location ~* \.(?:ttf|ttc|otf|eot|woff|woff2)$ {
   if ( $http_origin ~* (https?://(.+\.)?(domain1|domain2|domain3)\.(?:me|co|com)$) ) {
      add_header "Access-Control-Allow-Origin" "$http_origin";
   }
}

More sligtly configuration:

location / {

    if ($http_origin ~* (^https?://([^/]+\.)*(domainone|domaintwo)\.com$)) {
        set $cors "true";
    }

    # Nginx doesn't support nested If statements. This is where things get slightly nasty.
    # Determine the HTTP request method used
    if ($request_method = 'GET') {
        set $cors "${cors}get";
    }
    if ($request_method = 'POST') {
        set $cors "${cors}post";
    }

    if ($cors = "true") {
        # Catch all incase there's a request method we're not dealing with properly
        add_header 'Access-Control-Allow-Origin' "$http_origin";
    }

    if ($cors = "trueget") {
        add_header 'Access-Control-Allow-Origin' "$http_origin";
        add_header 'Access-Control-Allow-Credentials' 'true';
        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
        add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
    }

    if ($cors = "truepost") {
        add_header 'Access-Control-Allow-Origin' "$http_origin";
        add_header 'Access-Control-Allow-Credentials' 'true';
        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
        add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
    }

}

It's a fork bomb.

• :() - this defines the function. ":"" is the function name and the empty parenthesis shows that it will not accept any arguments
• { } - these characters shows the beginning and end of function definition
• :|: - it loads a copy of the function ":: into memory and pipe its output to another copy of the ":" function, which has to be loaded into memory
• & - this will make the process as a background process, so that the child processes will not get killed eventhough the parent gets auto-killed
• : - final ":" will execute the function again and hence the chain reaction begins

The best way to protect a multi-user system is to use PAM to limit the number of processes a user can use. We know the biggest problem with a fork bomb is the fact it takes up so many processes.

So we have two ways of attempting to fix this, if you are already logged into the system:

• execute a SIGSTOP command to stop the process: killall -STOP -u user1
• if you can't run at the command line you will have to use exec to force it to run (due to processes all being used): exec killall -STOP -u user1

With fork bombs your best method for this is preventing from being to big of an issue in the first place.

If a file has been deleted but is still open, that means the file still exists in the filesystem (it has an inode) but has a hard link count of 0. Since there is no link to the file, you cannot open it by name. There is no facility to open a file by inode either.

Linux exposes open files through special symbolic links under /proc. These links are called /proc/12345/fd/42 where 12345 is the PID of a process and 42 is the number of a file descriptor in that process. A program running as the same user as that process can access the file (the read/write/execute permissions are the same you had as when the file was deleted).

The name under which the file was opened is still visible in the target of the symbolic link: if the file was /var/log/apache/foo.log, then the target of the link is /var/log/apache/foo.log (deleted).

Thus you can recover the content of an open deleted file given the PID of a process that has it open and the descriptor that it's opened on like this:

recover_open_deleted_file () {
  old_name=$(readlink "$1")
  case "$old_name" in
    *' (deleted)')
      old_name=${old_name%' (deleted)'}
      if [ -e "$old_name" ]; then
        new_name=$(TMPDIR=${old_name%/*} mktemp)
        echo "$oldname has been replaced, recovering content to $new_name"
      else
        new_name="$old_name"
      fi
      cat <"$1" >"$new_name";;
    *) echo "File is not deleted, doing nothing";;
  esac
}
recover_open_deleted_file "/proc/$pid/fd/$fd"

If you only know the process ID but not the descriptor, you can recover all files with:

for x in /proc/$pid/fd/* ; do
  recover_open_deleted_file "$x"
done

If you don't know the process ID either, you can search among all processes:

for x in /proc/[1-9]*/fd/* ; do
  case $(readlink "$x") in
    /var/log/apache/*) recover_open_deleted_file "$x";;
  esac
done

You can also obtain this list by parsing the output of lsof, but it isn't simpler nor more reliable nor more portable (this is Linux-specific anyhow).

It is possible that the question should be: "System installation from the level and in place of already other system working".

On the example of the Debian GNU/Linux distribution.

1. Creating a working directory and downloading the system using the debootstrap tool.

_working_directory="/mnt/system"
mkdir $_working_directory
debootstrap --verbose --arch amd64 {wheezy|jessie} . http://ftp.en.debian.org/debian

1. Mounting sub-systems: proc, sys, dev and dev/pts.

for i in proc sys dev dev/pts ; do mount -o bind $i $_working_directory/$i ; done

1. Copy system backup for restore.

cp system_backup_22012015.tgz $_working_directory/mnt

However, it is better not to waste space and do it in a different way (assuming that the copy is in /mnt/backup):

_backup_directory="${_working_directory}/mnt/backup"
mkdir $_backup_directory && mount --bind /mnt/backup $_backup_directory

1. Chroot to "new" system.

chroot $_working_directory /bin/bash

1. Updating information about mounted devices.

grep -v rootfs /proc/mounts > /etc/mtab

1. In the "new" system, the next thing to do is mount the disk on which the "old" system is located (e.g. /dev/sda1).

_working_directory="/mnt/old_system"
_backup_directory="/mnt/backup"
mkdir $_working_directory && mount /dev/sda1 $_working_directory

1. Remove all files of the old system.

for i in $(ls | awk '!(/proc/ || /dev/ || /sys/ || /mnt/)') ; do rm -fr $i ; done

1. The next step is to restore the system from a backup.

tar xzvfp $_backup_directory/system_backup_22012015.tgz -C $_working_directory

1. And mount proc, sys, dev and dev/pts in a new working directory.

for i in proc sys dev dev/pts ; do mount -o bind $i $_working_directory/$i ; done

1. Install and update grub configuration.

chroot $_working_directory /bin/bash -c "grub-install --no-floppy --root-directory=/ /dev/sda"
chroot $_working_directory /bin/bash -c "update-grub"

1. Unmount proc, sys, dev and dev/pts filesystems.

cd
grep $_working_directory /proc/mounts | cut -f2 -d " " | sort -r | xargs umount -n

None of the available commands, i.e. halt, shutdown or reboot, will work. You need to reload the system configuration - to do this, use the kernel debugger (without the 'b' option):

echo 1 > /proc/sys/kernel/sysrq
echo reisu > /proc/sysrq-trigger

Of course, it is recommended to fully restart the machine in order to completely load the current system. To do this:

sync ; reboot -f

Major distribution kernels set the default value of /proc/sys/vm/overcommit_memory to zero, which means that processes can request more memory than is currently free in the system.

If memory is exhaustively used up by processes, to the extent which can possibly threaten the stability of the system, then the OOM killer comes into the picture.

NOTE: It is the task of the OOM Killer to continue killing processes until enough memory is freed for the smooth functioning of the rest of the process that the Kernel is attempting to run.

The OOM Killer has to select the best process(es) to kill. Best here refers to that process which will free up the maximum memory upon killing and is also the least important to the system.

The primary goal is to kill the least number of processes that minimizes the damage done and at the same time maximizing the amount of memory freed.

To facilitate this, the kernel maintains an oom_score for each of the processes. You can see the oom_score of each of the processes in the /proc filesystem under the pid directory.

cat /proc/10292/oom_score

The higher the value of oom_score of any process, the higher is its likelihood of getting killed by the OOM Killer in an out-of-memory situation.

If you want to create a special control group containing the list of processes which should be the first to receive the OOM killer's attention, create a directory under /mnt/oom-killer to represent it:

mkdir lambs

Set oom.priority to a value high enough:

echo 256 > /mnt/oom-killer/lambs/oom.priority

oom.priority is a 64-bit unsigned integer, and can have a maximum value an nsigned 64-bit number can hold. While scanning for the process to be killed, the OOM-killer selects a process from the list of tasks with the highest oom.priority value.

Add the PID of the process to be added to the list of tasks:

echo <pid> > /mnt/oom-killer/lambs/tasks

To create a list of processes, which will not be killed by the OOM-killer, make a directory to contain the processes:

mkdir invincibles

Setting oom.priority to zero makes all the process in this cgroup to be excluded from the list of target processes to be killed.

echo 0 > /mnt/oom-killer/invincibles/oom.priority

To add more processes to this group, add the pid of the task to the list of tasks in the invincible group:

echo <pid> > /mnt/oom-killer/invincibles/tasks

Salt at its most fundamental level is random data. When a properly protected password system receives a new password, it will create a hashed value for that password, create a new random salt value, and then store that combined value in its database. This helps defend against dictionary attacks and known hash attacks.

For example, if a user uses the same password on two different systems, if they used the same hashing algorithm, they could end up with the same hash value. However, if even one of the systems uses salt with its hashes, the values will be different.

The encrypted passwords in /etc/shadow file are stored in the following format:

$ID$SALT$ENCRYPTED

The $ID indicates the type of encryption, the $SALT is a random (up to 16 characters) string and $ENCRYPTED is a password’s hash.

Use the below commands from the Linux shell to generate hashed password for /etc/shadow with the random salt:

• Generate MD5 password hash

python -c "import random,string,crypt; randomsalt = ''.join(random.sample(string.ascii_letters,8)); print crypt.crypt('MySecretPassword', '\$1\$%s\$' % randomsalt)"

• Generate SHA-256 password hash

python -c "import random,string,crypt; randomsalt = ''.join(random.sample(string.ascii_letters,8)); print crypt.crypt('MySecretPassword', '\$5\$%s\$' % randomsalt)"

• Generate SHA-512 password hash

python -c "import random,string,crypt; randomsalt = ''.join(random.sample(string.ascii_letters,8)); print crypt.crypt('MySecretPassword', '\$6\$%s\$' % randomsalt)"