Archive for March, 2009

Unix: Keepin’ things regul’r w/ crontab

Sunday, March 29th, 2009

In Unix, to run a script at some regular interval, notate it in a crontab file. A crontab file is a list of programs that cron should run at specified times.

Cron itself is an OS service that runs scheduled jobs, those in any one of a number of crontab files on the machine. It is Unix daemon–It basically wakes up every minute, looks to see if any jobs need to be completed. If there are, it starts the job. If not, it goes back to sleep.

To find if cron is running on your system, type in, at the command line “ps -aux,” which will give you a list of all the processes running on the machine. Look for the name cron under the far-left column, called commands i.e.

root 4556 0.0 0.2 2100 888 ? Ss Feb17 0:01 /usr/sbin/cron

While there a few ways to get cron to execute a job (cron is another topic) , crontab is one way to get the job done. Like I said, crontab is basically a list of scripts, commands and programs that Unix can execute.

In Ubuntu (as w/ most other Linux distros), you edit the crontab file directly, by evoking it from the command line, i.e.:

#crontab -e

Listing jobs in crontab can be done by typing in:

#crontab -l

Anyway, when you edit, you will get a file. It may already have some jobs in it, i.e.:

# m h dom mon dow command

17 * * * * echo “hello”

In the first block above, the line (“# m h dom…”) is a header that is the key to explaining each of regularly scheduled jobs that will follow in subsuquent lines. m=minute, h-hour, dom=day of month, mon=month, dow=day of week, and command is the command or program that will be executed at the time indicated on the left.

Each job gets its own line. You add a job to be scheduled by adding a new line.

In the first five columns that specify time, “*” is null–means that column is unset. All the values are numeric and occasionally 3-letter abbreviations (i.e. week is 0-7 [0 & 7 = Sunday], though can also be sun “Sun”; month is 0-12 can also be name of the month. Hour is 0-23, with 23 being midnight).

After the first five entries are filled out, indicating when the job is to be run, the rest of the line is the job itself, expressed as a standard command line statement: It can be a command or series of commands that need to be run, or a program w/ the pathname.

In the above example, the command (“echo ‘hello’”) is run on the 17th minute of each hour. It prints the word “hello” on the screen.

With the day/week/month, you should specify on what time that job executes (If you run two times, say a day of of week and a day of the month, cron will run twice unless they fall on the same day). If you don’t specify what day/week/month, it will run every day, at the time you specify. You can also specify times -per unit, i.e., in the minute column, you can write “*/10″ to signify to run the job every 10 minutes.

(UBUNTU note: The User Geeks page says the crontab file is found in the /etc folder. Ubuntu’s own documentation advises you not to use this file–evidently it can be replaced by updates. Maybe it is only used for the configuration settings)


So, for instance, say I want to run a script, called “backup,” which backs up my files to another location. I want it to run, say, once a day (I don’t want to back up too often, in case I mess up a file, I can quickly retrieve). I’m usually never awake at 3:20 a.m., so I’ll specify that time each day. My new line in the crontab would look like this:

m h dom mon dow command
22 3 * * * /home/jobs/backup

If you want to run multiple commands simultaneously, use the “&&” between the two commands.


Getting a log file of your cron jobs

If you want a record of how things went, you can specify a (plain text) log file that can write out any results that would have otherwise be returned from the command line. You add the “>>” onto the end of your job, followed by the name of the log file, and its location:

30 16 * * * root /root/scripts/ServerBack >> /root/scripts/backup.log



NOTES:

*Ubuntu server does not initially allow user access to cron. In order to get Cron, you can either put the user name in a file called cron.allow, in the /etc directory, and create a cron.deny and not put that user name in that file.)

*If things aren’t running properly, check the var/log/syslog file for any error messages. For instance, my crontab entries did not work under root. When checking the log I found the

Mar 29 22:26:01 warehouse CRON[20531]: User account has expired

According to this article, when you lock a user account (so that it can’t be accessed externally), it also “expires” the password (Note: This isn’t a problem with Ubuntu out the box–it only happens when you unlock, then lock the root account). The log entry suggested running this command:

#chage -E-1 root

Which permanently unlocks the password. I have no idea *why* this works, but it has.

*For purposes of backup, you should track down the crontab files for each user (Again, this is not in the etc/ folder), so they can be saved. Beats rewriting them again when you set up a new server.

–Joab Jackson

And now, a word from our sponsor:





Unix: Removing outside Root access from Ubuntu

Saturday, March 28th, 2009

Every Unix box has a root account, or an account that has absolute control over the machine, its files and controls.

From all accounts, admins should use this account sparingly, only for cases where cross-system control is needed. You can make big mistakes (i.e. wipe out the system with a badly-worded command) from root. If you have work to do that can be done from a user account, use that instead.

Ubuntu has taken this precaution to the next level. When you install Ubuntu, it disables the root account. If you want to do root commands, you issue them from your account through the “sudo” command. basically sudo allows you to issue commands from another user’s account (assuming you have the password).

The idea behind this is that not only will it limit the mistakes you may make, but also reduce the attack footprint of your server. A cracker, trying to get in via SSH, doesn’t even get the option to guess the password to the root account to gain entry, because root can’t log in from SSH at all.

Now, like many old Linux codgers, as soon as I set up Ubuntu, I enabled the root account, mainly because filling in a password every time I wanted to do something in root was a pain. But I just learned that sudo actually has an option (sudo -i) that will allow you to work within a shell of a user account entirely–this means you evoke sudo once and then every thing you do from the command line is done from the account you’ve sudo’d in from, such as root. which is pretty neat, and eliminates the need to be able to access root from the outside (i.e. via SSH).

But if you already enabled root on your Ubuntu, how do you un-enable it? Simple, from the root account, use the psswd command to “lock” the account, i.e. make it inaccessible from outside users. “passwd -l root”

–Joab Jackson





Unix: Removing directory access from Web sites

Wednesday, March 25th, 2009

I learned this from a SANS security class: Don’t expose directory listings for public folders on your Web site. You know, if you type something like “http://www.misc.com/Flounder/” and by typing the end “/” you can get the contents of the directory–including subdirectories. Not good to reveal information you don’t want out, such as non-public directories.

The easiest way to combat this is to simply put an index.html in each directory that is being exposed. It can be an empty page. That way when someone does some path guessing, they are met with this blank page.

Here’s a simple all-black page I used:

<html>
<head>
<title>The Void
<Style type=”text/css”>
body {background-color: #000000}

</head>
<body>
<pre>

</pre>
<center>
<a href=http://www.joabj.com>Home

</center>
</body>
</html>

--Joab Jackson





Unix: Getting the most from the “ls” command

Thursday, March 19th, 2009

You can learn a lot from the the Unix “ls” command, once you know all the options. If you come from the world of Microsoft DOS, then ls is Unix’s version of “dir.” From the command prompt, type in

#ls

will return a list all the files and directories in the current directory are in.

As in many Unix commands, there is a great deal you can do with the ls’s additional options. Options are noted by the “-” sign, followed by a letter deisgnating which option to use. Note which case is used for the option, as they are case-sensitive. Here are some useful options:

#ls -l
….This of the verbose listing, offering, in addition to the name of each file/subdirectory, alkso offers (in this order): user permissions, the user permissions, owner, and group that the item belong to.

size of file, last modified

#ls -a
….Lists hidden files, those files that start with a “.” . Use this for finding system files.

#ls -F
….Appends each entry with either a “/” (to indicate it is a directory), a “*” (executable file) or “@” to show it is a symbolic link.

#ls -G
….Lists the group each file is owned by.

#ls -i
….Lists the innode (storage location on the disk) for each file.

#ls -L
….Lists the files referenced by the symbolic link, ratherthanthe names of the links themselves (if using symbolic links. See future entry on symbolic links).

#ls -m
….Separates each file and directory with a comma.

#ls -n
….Lists each file along with group identifying (GID) and user-identifying (UID) numbers instead of the group/owner names, respectively.

#ls -o
….Show owner with names.

#ls -p
….Indicate the directories by appending “/” onto the end of their names.

#ls -r
….List in reverse alphabetic order.

#ls -R
….List the contents of all the subdirectories.

#ls -s
….List the size of each file in blocks.

#ls -t
….List files in order of when they were modified, starting from the newest (but no dates).

#ls -u
….List files in order of when they were accessed, starting from the newest (but no dates).

#ls -t
….List one per line.


Some end-notes:

1. Using multiple options at once: Multiple options can be used together, i.e.

#ls -u1
…lists the files in order of when they were last accessed, one per line.

2. Piping ls results into a text file: Normally, when you use ls, it drops the input onto the screen. But you can also send the results (or “pipeline” them) to a file, using “>.” i.e.

#ls -a > [nameoffile]

If no file with that name exists, a new file is created with that name. The results can be piped into other commands as well. Check for future entries on pipelining.

3. Wildcards: Wildcards can be used to pick out only those files you are looking for. I.e.

#ls S*
…Will return only those files that start with the letter S.


A full list of working commands can be found on the ls man page, here. More explanation can also be found here. Also, check out Unix in a Nutshell by Daniel Gilly (O’Reilly).

–Joab Jackson





Unix: managing users

Thursday, March 19th, 2009

Adding a user is done from the command line, with the command “useradd” from the root account…

>useradd [new user name]

useradd has several options. For instance, you can specify the working directory of that user (the default is a subdirectory with the user’s name in the home directory). You use the command -m and the name of the directory with no spaces after that, i.e.:

>useradd [NewUser] -m/home/somewhere/somewhereelse

There is also the command to add in the password, though I’d advise to not define the password as part of adduser procedure. Do it as a separate command. The log-on seems to not recognize passwords done from this command. Maybe because it doesn’t use the hashing procedure.

Other commands in the user management set include “usermod” and “userdel.” usermod modifies the various attributes of the user account,m using the same option set for useradd and userdel deletes the person’s account. No need to do examples, follow the format above.

More info on managing users in Unix can be found here


A special section on PASSWORDS
You can add or change a password from the command line passwd (please to note missing “ord”). You can do your own account’s password; root can do anybody’s.

So here is something to clarify the typically obtuse man pages: You enter password command the user name and then hit ENTER

>passwd [hit enter]
Enter new UNIX password: [enter the new password here. Hit enter]
Retype new UNIX password: [Retype new password, hit enter]

More about passwords here.

–Joab Jackson





Traceroute: Finding a lost connection.

Wednesday, March 18th, 2009

Recently, my Web site seemingly stopped responding. Or at least it seemed that way from where I was, about 100 miles away at the time. I could not SSL in, nor could I get the Web page for the site. Even the Ping command did not work. Was my server down? Or was the network down?

I used the traceroute command to determine where the problem spot is. As it turned out, Traceroute did not solve the problem entirely, though it allowed me to define the troublespot with more precision.

Traceroute maps each hop along the journey from my computer to the end point that computer is trying to reach. On Windows, the command is “tracert [host name].” On Unix/Linux, it is “traceroute [host name].”

These were the results:

Tracing route to joabj.com [75.148.24.2]
over a maximum of 30 hops:

….snip first few hops…

5 12 ms 12 ms 12 ms so-15-0-0-0.RES-BB-RTR1-RE1.28.81.130.in-addr.arpa [130.81.28.248]
6 13 ms 12 ms 12 ms 0.so-1-2-0.XL3.IAD8.ALTER.NET [152.63.37.117]
7 14 ms 12 ms 11 ms 0.ge-6-0-0.BR2.IAD8.ALTER.NET [152.63.41.149]
8 12 ms 12 ms 11 ms te-11-3-0.edge1.Washington4.level3.net [4.68.63.169]
9 14 ms 14 ms 13 ms ae-1-69.edge1.Washington1.Level3.net [4.68.17.16]
10 14 ms 14 ms 15 ms COMCAST-IP.edge1.Washington1.Level3.net [4.79.231.14]
11 14 ms 15 ms 14 ms 68.85.130.53
12 15 ms 14 ms 14 ms 68.85.130.54
13 * * * Request timed out.
14 * * * Request timed out. (repeated until hop number reached 30)

Trace complete.

Each successive line represents the next hop on the network. In the above set of data, the first column is the count of nodes from the client making the request. The following three times (in milliseconds) indicates the the time that node took to respond to the traceroute request (about 500 milliseconds is the upper-end of acceptable response time. And the last column is the Internet Protocol (IP) number and, if available, the domain name of the node at that location.

As this page helpfully explained, it appeared as if a node at least one hop away from my own server was not not responding (75.148.24.2).

This was indicated by the * * * in the output. The asterisk means that the node did not respond to the traceroute request in a timely manner. A single asterisk can be typical, especially for a busy server, though all asterisks indicates trouble is afoot.

What was odd was that my own server was not the last IP number listed. So it could be that it was down, or it could be that there was another hop. As the tutorial suggested, I did a traceroute from another location, in this case from
Network-Tools(A whole list of others could be found here). From this location, it appeared as if my server was responding fine, but the hop before it was not responding, I.e.:

Hop (ms) (ms) (ms) IP Address Host name
1 28 6 8 72.249.134.177 -
2 7 6 8 8.9.232.73 xe-5-3-0.edge3.dallas1.level3.net
3 9 6 6 4.71.198.14 comcast-ip.edge3.dallas1.level3.net
4 29 30 42 68.86.85.254 pos-0-13-0-0-cr01.atlanta.ga.ibone.comcast.net
5 47 45 42 68.86.85.238 pos-0-12-0-0-cr01.mclean.va.ibone.comcast.net
6 46 48 59 68.85.130.53 -
7 54 48 47 68.85.130.54 -
8 Timed out Timed out Timed out -
9 59 50 53 75.148.24.2 www.joabj.com

Trace complete

So, whatever that hop is between 68.85.130.54 and my own site (75.85.130.54) was not responding. But my own site was! Most peculiar.

Adding to the mystery even further, I found I could access my Web site just fine, using an anonymous public proxies (public proxies are third-party sites you can go to visit other Web sites, without either that site or your own network administrator knowing the IP details of either party). This meant that other parties could access my site, but no computer from the IP address I was could.

This other site notes that the no information coming back from a hop (“* * *”) means only that that hop doesn’t respond to traceroute requests, not that it is not operating correctly. It could not be returning information for security reasons, for instance. This would not be the case if it were the last hop that was not responding in the traceroute—this is why it is important to do traceroute from alternate locations. (And this page points out that sometimes timed-out requests do not, at least in some cases, represent, nodes; rather they are artifacts of the traceroute reporting mechanism.)

So, summary (for now): Something at the location one hop away from my server is blocking access to that server, though only for my own remote location. According to this page, this may be my IP provider (Comcast) blocking any traffic from my own IP address (maybe because of the large number of files I had been transferring between the two locations).

The good news that I am working from a location with dynamic IP, so getting another IP address address should do the trick. (Update: I regained access from this location the next day).

–Joab Jackson





Unix: Changing file owners

Monday, March 16th, 2009

In a Unix system, every file & directory has an owner.  Initially, it is the account that created the file that is designated as the owner. This can be changed through the “chown” command.

chown is important to know insofar that Unix is designed so that programs and files can only be executed/opened by designated users (& the root command). When something is not executing properly, take a look at the ownership permissions  (which can be done through the ls -o command, at the command line, which shows the names of the files and the owners of those files)

At the command line, the command is formatted like this:

>chown [options] [the new owner] [the files affected]

i.e.

>chown -R robert taxes

The options are:

-R changes all the ownership permissions down through the nested subdirectories.

-h changes the ownership of the symbolic links, rather than of the files themselves.

More info on the manual page here.

–Joab Jackson





MySQL: Updating passwords

Thursday, March 5th, 2009

Updating a password can be done within mysql, by root, thusly:

mysql>use mysql;
mysql>update user set password=PASSWORD(‘[Password]‘) where user=’[Name of user]‘;

Note, to change the password from the account you are already in, you do not need to specify user i.e.:

mysql>set password=PASSWORD(‘[password]‘)

The second designation of PASSWORD is actually a function call, one that encrypts, or hashes the password you supply.

Also, keep in mind that, after any password updates you must flush privileges:

mysql>flush privileges;

And jumping back out to the command line and restarting MySql is not a bad idea, either:

mysql> quit
# /etc/init.d/mysql stop
# /etc/init.d/mysql start

Info taken from here and here, and, if you really get into trouble, here.

–Joab Jackson





MySql: Adding or dropping a new user

Wednesday, March 4th, 2009

Adding a new user is a three-step process. First you add the user account, and then you grant privileges to whatever databases that person will be using.

1. To add a user, you use the Create User command. Log onto Mysql as root, then, at the prompt, type:

mysql>create user [name of new user];

[MYSQL BUG: The documentation says you can add password onto this line, i.e. "identified by '[password]‘ but YOU CAN’T!!! You could spend all morning wondering why your new client can’t log in because of this SNAFU. Instead, you have to add the password in a separate command (see the section on adding passwords). So, step 2 is:

2. Add password for new user (see separate blog entry).

3. To grant database privileges to the user, you use the grant command. First you stipulate the privileges you want to grant, specify the name of the databases these privileges would apply to, and finally whom all this should apply to. Intuitively backwards.

mysql>GRANT ALL ON [databasename].* TO ‘[username]‘@’[somehost]‘;
mysql>GRANT SELECT, INSERT ON *.* TO ‘[someuser]‘@’[somehost]‘;

Note, that somehost specifies where the host is working from, either remotely (over a network) or on the terminal of the machine where mysql resides (localhost). You can also specify specific IP numbers, I think. If you leave @ off altogether, mysql assumes the user can access these privileges from anywhere.

Also note that using a database usually involves working with multiple files, all under the name of the database but with different suffixes. So that it is usually good to use a wildcard for the suffix of these files for grant privileges.

[MYSQL BUG: The '%' wildcard is deceptive, insofar as it does not grant privileges from the local host, as the documentation states. Instead, you have to grant a set of privileges especially for the localhost, i.e. '[someuser]‘@’localhost’.]

* * *

Dropping a user account can be done thusly:

mysql>drop user [Name of user]

Taken from here

–Joab Jackson