Week 8, CSc 103

Using and Setting up a Database

Week of October 29 - November 2

Comprising:

  1. What kind of data would you like to store?
  2. Crash course in using SQL
  3. Accessing your database from Python
  4. Storing data using a CGI script - and querying it. See my references for each cgi language for syntax and semantics

Think about your final form

So far you have been constructing rather arbitrary forms, please give some thought to what sort of form(s) you would like on your final web site. If you are modeling a store, it might be either orders, or items you have for sale, if you are inviting poetry submissions, it could be poems, their authors, and contact information, etc. Think about which fields are optional, and which should be required (and therefore subject to javascript checks.) For each field, what type is it? Integer, float, date, character string? How many characters maximum?

Crash course in SQL

Databases store data in tables, consisting of rows and columns. Columns have names, and store data of the same type. We have PostgreSQL database management system (DBMS) installed on osiris (and also on cs-linux).

When it comes to accessing the data, there is a big advantage over just appending to a file: the data is structured, and there are lots of possibilities for querying the data selectively. You can ask for certain columns, and  give criteria for which rows you want. You can sort the results as well.

There is already a database "timetable" on Osiris, with table "students", having columns given, family, major, linux and email.
Also a table likes, with columns linux and activity. (Linux would be the code of some student, allowing the tables to be "Joined"). Both these tables have some data from last year, and you can insert some of your own.
There is also a table enroll, with no data yet.
You may connect to this database and practice queries. Your linux username is also your postgres username.

How to connect

  1. In the Linux lab, use the psql interactive terminal, and give your password.  psql -h osiris timetable
  2. by ssh (or Putty) to osiris, and use the psql interactive terminal. No password required.  psql timetable
    You can start typing SQL Select queries immediately.
  3. In the Linux lab, use Pgadmin3 (blue elephant icon) (need to create a connection to osiris, with username and password), Then start clicking.
  4. From anywhere with a web browser: http://osiris.ubishops.ca/phpPgAdmin
 Without further ado, here is a sample interactive terminal session. I did this on my home computer actually. You can connect to an existing database on osiris, and I (will) have created one named after you (same as your user name), of which you are the owner, for your course work.
[jensen@goatherder 315lab]$ psql fiddle
Welcome to psql 8.0.6, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit

fiddle=> create table faddle(
fiddle(> name varchar(20),
fiddle(> age integer,
fiddle(> poem text);
CREATE TABLE
fiddle=> insert into faddle values('Lin',39,'Roses are red\nViolets are yellow');
INSERT 18827 1
fiddle=> select * from faddle;
name | age | poem
------+-----+---------------
Lin | 39 | Roses are red
Violets are yellow
(1 row)

Insert

SQL is not case sensitive. The form of the SQL insert statement is:
INSERT INTO faddle VALUES ('Lin',52,'The rain in Spain can''t fall')

INSERT INTO faddle (name, poem) VALUES ('Lin','The rain in Spain')
The result returned from executing an insert is either 1 (1 row inserted) or an error condition.

Warning: Single quotes are used for character data. Input may have an apostrophe, it must be doubled before using in SQL: eg. cat's cradle needs to be 'cat''s cradle' otherwise there will be syntax trouble, or worse (neglecting this leaves you open to nasty database attacks.)

Select

To query a database, one uses a select statement, in the form:
SELECT columns
FROM table
WHERE condition;

Examples:

* means "all columns. You do not need a WHERE clause if you want all rows.
SELECT *
FROM students;

SELECT given, family, email
FROM students
WHERE family = 'O''Hara';
For example, to find the names of all the authors whose poem contains the word 'rose', write
SELECT name 
FROM faddle
WHERE poem like '%rose%'; -- % matches any number of any characters
Only this won't fond my poem, since I wrote 'Rose'. One could use "ilike" for a case-Insensitive matching.

Join

A select statement "from" two (or more) tables joins (sticks together) rows from each that match, according to some condition. Here we want all the students named 'Alice', matched with their activities. We also want to sort, ordering by activity, so that all the Alice's who like croquet will be found together.
SELECT given, family, activity
FROM students, likes
WHERE students.linux = likes.linux
AND given = 'Alice'
ORDER BY activity;

Update and Delete

The other SQL data manipulation statements are UPDATE and DELETE. You will appreciate by now that a database makes it easy to maintain a collection of data. If a friend moves, or drops dead, you can modify your data easily, compared to having your CGI program search and rewrite a file similar to your log file.

Warning: Both of these need a WHERE clause, lest you alter ALL the rows of a table. Each of these examples should only affect 1 row.
UPDATE likes SET activity='drinking tea' 
WHERE linux='jdoe' AND activity='drinking beer';
DELETE from likes WHERE linux='jdoe' AND activity='extreme bartending';

 Connecting from a PHP program

Use the PostgreSQL functions, they all start with pg_ 

A result set is returned when a select statement is executed. It contains the data from a table of rows and columns, usually smaller than the table in the database, because you asked for particular data. A PHP program must then loop through all the result rows, and process (in your case, print) the column entries.

A query can also be an update statement (either insert, update, or delete). Such a query does not return a result, instead it modifies the database.

You connect to a database (on osiris), do any number of 'queries', look at the results of select queries, and close the connection (or else it is closed at the end of the script.) Here is a commented example:

$conn=pg_connect ('dbname=timetable') or die ('Connect failed ');

$query = "SELECT * from courses where prof='$profname'"; //from form
$result = pg_query($query) or die("query failed".pg_last_error());
print pg_num_rows($result) . " rows found for $profname<br>\n";

while ( $line = pg_fetch_assoc($result) )
foreach ($line as $key => $value)
print "$key = ".htmlentities($value)."<br>\n";

/* Free resultset */
pg_free_result($result);

$query2 = "INSERT into students (given, family, linux) values ($1,$2,$3)";
$result = pg_query_params($query2, array($giv, $fam, $linux)) or die("query failed".pg_last_error());
// (could fail on duplicate primary key )
print pg_affected_rows ($result) . " rows inserted for $linux<br>\n";
// note: use this, not num_rows, for insert, update, delete queries

pg_close($conn);
You will note the 'die' syntax if anything goes wrong. This will show you the error message from postgres.

Putting this together

After this, it shouldn't be too hard to include database operations in your CGI program. First get the form results, then construct the SQL query, and send it to the database, then print the results in whatever form you desire: a list, a table, as values of <input> fields ...

Little catch: Your cgi program runs as apache, who is also a database user. You must grant permissions on your table to apache, by using psql:
GRANT SELECT,INSERT,UPDATE ON tablename TO apache;
(I usually don't grant DELETE permission, too easy to lose information! Then I do deletes myself, using psql.)

Goal for this week:

A PHP program which, given (part of) an activity, will find all activities containing that string (use ILIKE '%$find%') and the people who like them.

By the end of next week, I would like you to set up your own database (phpPgAdmin should be helpful) and have a form whose data is inserted into your database using a php program. If possible, also a search page: Typically one input field, and the search is based on that (In the WHERE clause).

Prepared by Lin Jensen, 23 May 2006. Revised 28 October 2007