Showing posts with label SQL. Show all posts
Showing posts with label SQL. Show all posts

Wednesday, June 19, 2013

Quick Tip: Oracle Sqlplus - changing prompt

Oracle SQL*Plus has several settings that can be customized using SET command. When you run SQL*Plus, typically the following (boring?) prompt greets you.

SQL>

If you ever want to change this to something more interesting, there is a SET option for that.

SET sqlprompt 'sqlplus:&_user@&_connect_identifier > '

This will be shown as,

sqlplus:<user>@<SID>>

For e.g.,

sqlplus:scott@hr>

Where scott is a user id logged into hr database.

References

http://docs.oracle.com/cd/B28359_01/server.111/b31189/ch12040.htm
http://docs.oracle.com/cd/B19306_01/server.102/b14357/ch12017.htm#i2675128

Wednesday, June 12, 2013

Oracle: Commit and Forward Slash

Further to my post earlier about DDL and Forward Slash, I would like to explain Forward slash's role (or lack thereof) in other scenarios. One such scenario people seem to get confused with is COMMIT. They seem to want to find some type of connection between the two. Let me say it upfront, there is no connection between COMMIT and "/".

Forward Slash is simply a SQL Executor and it happens to show up in various situations. This combined with SQL*Plus's own quirks seem to imply certain other functions for this character. Let me repeat: it's just an executor - short for RUN command in SQL*Plus.

One reason it could be construed as connected with COMMIT is that it may often be the last character in a file. When you run SQL*Plus in batch mode (running a script in a file), it typically exits after the last line (on *nix. It doesn't DoS/Windows). And the SQL*Plus's own behavior is to COMMIT a transaction when exiting. (Why? See here for a nice discussion about this.) So, it's the exit that committed not the "/". But. you can see why some newcomer to Oracle who just inherited some scripts, may think that "/" (being the last statement) actually committed the transaction!!!

The other reason could be DDL. DDLs are implicitly committed. (Actually it commits before and after the DDL statement itself, so beware if you are mixing DMLs and DDLs).

Now, typically a DDL could be ended with "/". For e.g.,
INSERT INTO department(100, 'HR');
CREATE TABLE employee(
emp_id NUMBER,
emp_name VARCHAR2(40)
/

In the above case, department  table will have the entry, even if the CREATE TABLE below failed. In generla, any DML (like INSERT/UPDATE/DELETE) before a DDL, DML would have been committed, even if the DDL itself failed. Since "/" is the last statement here, one could think, "/" did it!!! Another strike against "/".

So, trust me folks! "/" doesn't do anything else, except sending a SQL to Oracle in SQL*Plus. It's just a shortcut for Run Command. But, depending on the situation, it may look like it's doing something more. In such cases, analyze the SQL and try to rearrange them. Try to keep DMLs separate from DDLs to avoid surprises. And of course, set some standards for punctuations and stick to it to avoid any undesired effects.

Oracle: DDL and Forward Slash

In this blog,  I write about various technologies I come across. Only a few topics seem to get a lot of attention. One of them happens to be Forward slash in (Oracle) SQL. I've already posted about this here and here. When I look at the search terms that lead to my site, I sense a lack of understanding of it's usage. A search through the web quickly reveals that forward slash has been a source of confusion even among some of the experienced Oracle SQL developers 1. Based on the search terms, I can see the newcomers are often getting confused by the SQLs left for them by their predecessors and are looking for a clear explanation. "/" is often mixed in various statements, such as DDL, COMMIT etc and this leads to confusion about the statement itself. I will try to explain the role of "/" (or lack thereof) in each different scenario in separate posts.

One of the search term that I get a lot is "DDL and Forward slash" or some variation of it. Forward slash has nothing to do with DDL per se. If you saw my other posts, it is just a statement executor (RUN command for both SQL and PL/SQL). Unfortunately, semi-colon doubles as statement terminator and executor for SQLs (poor choice of shortcut) in SQL*Plus. There lies the confusion. Thus, the CREATE TABLE or DROP VIEW statements can be terminated by a semi-colon as well as "/".

For e.g.,
CREATE TABLE employee(
emp_id NUMBER,
emp_name VARCHAR2(40));

and
CREATE TABLE employee(
emp_id NUMBER,
emp_name VARCHAR2(40))
/

are identical as for SQL*Plus is concerned. But, as a matter of preference, people tend to use "/" in DDL.

But this is not the case in other DDL statements that involve PL/SQL:
CREATE OR REPLACE FUNCTION f_get_employee(a_emp_id NUMBER)
RETURN VARCHAR2
AS
v_emp_name VARCHAR2(40);
BEGIN
SELECT emp_name
INTO v_emp_name
FROM employee
WHERE emp_id = a_emp_id;
END;
/

Here "/" is a must, as the above statement is one PL/SQL statement (from CREATE to END;). Remember, PL/SQL block ends with END; and it has to be followed by "/" to be executed 2 (see notes below).  Thus some DDL statements (those that create programming constructs like packages, procedures, triggers) require "/" at the end. This, I think, led the newcomers to believe there is a link between "/" and DDL!!! There is no connection! You cannot generalize the use of "/" for all DDLs. For e.g., if you tried to do this (semi-colon followed by "/") in a CREATE TABLE statement, you will have unexpected behavior:
SQL> CREATE TABLE employee(
2 emp_id NUMBER,
3 emp_name VARCHAR2(40));
Table created.SQL> /
CREATE TABLE employee(
*
ERROR at line 1:
ORA-00955: name is already used by an existing objectSQL>

In this case, the semi-colon followed by slash is a bad idea.

In a nutshell, use only semi-colon or slash at the end of a SQL (DML or DDL) and semi-colon followed by slash at the end of a PL/SQL. See here for the difference between SQL and PL/SQL).

As a personal choice, I normally use "/" for all DDLs (SQL or PL/SQL) and semi-colon for all DMLs. You can have one DDL per file in which case, the "/" is the last character in the file. You can also have multiple DDLs in a file, each separated by "/".  Try not to mix DDLs and DMLs in the same file! This may lead to more confusion and errors!

References

http://www.orafaq.com/faq/what_are_the_difference_between_ddl_dml_and_dcl_commands

http://docs.oracle.com/html/A90108_01/sqcmd.htm

Notes:




1. See here for e.g., https://forums.oracle.com/thread/1020117

2. When I say "executed", I mean the PL/SQL block that creates the function/procedure is executed - meaning the program is compiled and stored in the database. (Hence the name stored programs.) The actual program execution is done by
SELECT <function name> FROM dual;

OR EXEC <procedure name>

Monday, April 15, 2013

Oracle SQL - UNION ALL

Oracle SQL - UNION ALL

Here is another TechTarget article about Oracle SQL. This time about using UNION ALL to combine results of 2 SQLs to sort numbers.

Saturday, February 2, 2013

SQLite Database

I've been working with different types of Databases over the years from dBase to B-Trieve to Oracle enterprise database. Along the way, I looked at or picked up quite a few - Integra SQL, Informix, MS Access, MySQL, SQL Server(s), Watcom SQL/Sybase SQL Anywhere, DB2, DB2/400 etc. While SQL databases are good, I miss the simplicity of dBase or B-Trieve files. After many years now, I think I found the lost love now!! SQLite!

In my intro I talk about "beautiful software". This is how I felt about this software. Installation is simple, you just download and unzip 1 EXE - sqlite3.exe. (or install in Linux using apt-get or yum). Done! When you are ready to create a database, you type sqlite3 <database name>. That's it!!! You want to work with the same database again, just type,
sqlite3 <database name>

The file size is what amazes me - it's a mere 50k. They've really packed a lot inside it. For one, it supports almost full (basic) SQL. I was able to actually run SQLs meant for Oracle in Sqlite. The command line utility achieves what other big databases didn't. SQL*Plus can learn a lot from this tool. And it's highly portable - in fact, I created a database in windows at work and brought home to Linux. It works like a charm.

For one, they differentiated the tool commands by .prefixing with period (.). So, you want help, just type .help. You want to see the tables in the database you just opened, just type .tables.

What impressed me with the tool, other than the simplicity is, that it provides table export imports naturally. Commands?
.import <filename> <table>

What about exporting table or query output? Just use,
.dump [<tablename>]

.output <filename>

Don't you want to export to CSV or HTML etc.? Sure, just use .mode. Then, you want a different separator? Use .separator. (Though HTML mode only dumps the tags for table rows, as it's output is often embedded in CGI outputs, but you can always add the missing tags, right?). Have you tried exporting a table from Oracle SQL*Plus? It's like extracting teeth, so we end up writing programs/procedures to do it.

True, Oracle (and other databases) is much larger and has a lot more bells and whistles, but for a simple (personal) database task, SQLite is the natural choice. Granted, this is just file based database, but smaller tasks this is a great tool to have in your toolbox. My specific task this time, is to import the log files into a database table, so I can query and procedure usage reports and charts for our recently re-factored web app. In the past, I used MS Access for this type of activity, but with a slow machine I have, Access doesn't fair well with hundreds of thousands of log records.

I've only scratched the surface with SQLite. It seems to be used in embedded applications. There is an API available to work with it C/CPP. Python (Perl, PHP etc) come with native support for SQLite. It's a pleasure to work with SQLite in Python. More on this later. Apart from these, SQLite offers extensibility by allowing to load (.load) external modules. You can also use Virtual Tables (like Oracle External table) to access a CSV file directly.

I can go on and on, but I will stop ranting and raving about the greatness of this software. If this post made you want to look up more on SQLite, please try the links below.

  • http://www.sqlite.org

  • http://en.wikipedia.org/wiki/SQLite

  • http://www.sqlite.org/whentouse.html

  • http://www.sqlite.org/selfcontained.html

  • http://www.askyb.com/sqlite/learn-sqlite-in-1-hour/

  • http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles

  • http://code.btbytes.com/2010/09/16/sqlite-csv-loading-using-virtual-table-extension.html

  • http://docs.python.org/2/library/sqlite3.html

  • http://www.sqlite.org/vtab.html

  • http://www.drdobbs.com/article/print?articleId=202802959&siteSectionName=database

  • http://www.yolinux.com/TUTORIALS/SQLite.html

  • http://robert.accettura.com/blog/2008/02/26/and-the-winner-for-most-embedded-db-is-sqlite/


 

Saturday, October 20, 2012

Gotcha - Oracle: Null

NULL essentially points to an "undefined" value in the database. It helps to conserve space in database columns. It's useful when we want to store "nothing" in a column. NULL is defined in ANSI, so all databases including ORACLE implement them. But, ORACLE differs from ANSI standards in dealing with NULLs. Some of these could be real gotchas for developers coming from other databases.

Gotcha 1 - Empty String is the same as NULL in Oracle SQL

In case you didn't know, empty String (also known as null string)('') and NULL are considered the same in Oracle. This is not ANSI standard and is a real gotcha for developers coming from other databases, where they are different.

Here is a SQL I found in one of the programs I am supporting:
SELECT * FROM employee WHERE NOT((employee.dept_nbr IS NULL) AND (employee.dept_nbr = '') );

In Oracle, the above Where condition is *not* really checking for 2 different things. Seems to me, the developer wanted to make sure it was not empty and it was not null either. This is because, he/she didn't realize Null String is essentially same as NULL in Oracle.

When in doubt, I always try a simpler query than original. Oracle system tables and the built in table DUAL come in handy for those queries. Here is a sample query that shows empty string is Null.
SELECT 'empty string is null' FROM dual WHERE '' IS NULL;

Another Gotcha - a bug in the SQL above


The above example had another bug. if the NULL was not the same as '', the above query would have failed completely!! (How can same field be NULL AND something else (here null string ('')) at the same time?). The above condition must have been
SELECT * from employee WHERE NOT((employee.dept_nbr IS NULL) OR (employee.dept_nbr = '') );

--
OR even below SQL:

SELECT * from employee WHERE (NOT(employee.dept_nbr IS NULL) AND NOT(employee.dept_nbr = '') );

Gotcha 2 - Checking for NULL

Did you notice "IS NULL" there? When you check for a value being NULL, you cannot use equal to (=). You have to use IS or IS NOT. This is yet another gotcha for many new developers to Oracle. Strange thing is Oracle will not complain if you entered '' = NULL in the above SQL. It simply won't find any rows!! Who better to explain this than Ask Tom?









[caption id="attachment_585" align="alignleft" width="329"] Oracle Select "IS NULL"[/caption]


[caption id="attachment_587" align="aligncenter" width="412"] Select equal to NULL results in "no rows found"[/caption]

If you are coming other databases, this may be a surprise to you (SQL Server for e.g., allowed = NULL check). It's a real gotcha in Oracle, because Oracle won't complain if you said "= NULL", but the query may not work as expected. A query may return less number of rows than expected, because of NULLs in some fields in the where condition. For e.g., below SQL will return only 2 rows (I expected the row with NULL to show as well).
SELECT emp_nbr, dept_nbr, first_name || ' ' || last_name || '(' || dept_nbr || ')' AS employee
FROM employee WHERE employee.dept_nbr IN (NULL, 100. 200);

The above WHERE condition must be written as,
WHERE employee.dept_nbr IS NULL OR employee.dept_nbr IN (100, 200);

Another way of doing this is to use the NVL function in Oracle. NVL translates NULL value to the value passed in.
WHERE NVL(employee.dept_nbr, 0) IN (0, 100, 200);

Here NULL is translated to 0 and this we can check in equality or IN condition.

Gotcha-3 Concatenating NULL (Empty String)


When you concatenate text fields together, if any of the value is NULL, the result will *NOT* be NULL in Oracle. In other databases (for e.g., SQL Server, mysql), String + NULL results in NULL. In Oracle, only NULL + NULL results in a NULL.
SELECT emp_nbr, dept_nbr, first_name || ' ' || last_name || '(' || dept_nbr || ')' AS employee
FROM employee WHERE (employee.dept_nbr IS NULL);

In this case, if you expected NULL, you are in for surprise. The concatenation actually works. Only the NULL values will be missing. (For e.g., this will return Sam Varadar() based on the sample data in the screenshots above). Though, Oracle is not promising this in the future versions. See here.

Gotcha-4 Nulls and Indexes


Another hidden gotcha in this is that when we a query uses an index, we are implicitly applying Equality checks. If a column that is in an index has NULL values in some rows, then those rows will not be indexed and thus won't be picked up!

This also means a primary key column cannot have NULL values. This is because a primary key column value identifies each row uniquely and thus they have to be in the primary index on the table.

Other NULLs


We saw above, concatenation of strings works even if one of the values is NULL. This is because, a NULL is also considered an Empty String. Other operations may not be as kind:

For e.g., Adding NULL to a Number will result in NULL.
SELECT 1 + null FROM dual;

In PL/SQL NULL is a NULL statement - a statement that does nothing. It is often needed in control blocks, like if-then-else where we want to leave the if or else part empty, but PL/SQL syntax doesn't allow this.
IF (dept_nbr = 100) THEN
dbms_output.put_line('Inside IF block');
ELSE
NULL; -- Do nothing
END IF;

When sorting a query result using ORDER BY, you can sort rows with NULLs to come FIRST or LAST (default).
SELECT * FROM employee ORDER BY dept_nbr DESC NULLS FIRST;

Functions to deal with NULLs

Because of the unpredictable  behavior of queries around NULL values, it's often better to translate NULL into something meaningful, so equality (or inequality) tests can be done without worrying about NULLs.

We showed the use of NVL above. It translates NULL into a more meaningful value, 0. NVL2 is a similar function that returns one value for NULL and another for Non-NULL values. COALESCE is another function that could be passed in a series of comma separated values and it will return the first non-NULL value in the list.

These functions can also be used while creating index on a column that may have NULLs and thus help with performance also. See here for more on this.

Other functions include NULLIF, DECODE etc. See here for a nice discussion of the functions related to NULL.

References

  1. http://psoug.org/reference/null.html

  2. http://stackoverflow.com/questions/203493/why-does-oracle-9i-treat-an-empty-string-as-null

  3. http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:17320984423926

  4. http://www.sqlines.com/oracle/string_concat

  5. http://www.oracle-base.com/articles/misc/null-related-functions.php

  6. http://thinkoracle.blogspot.com/2005/06/nulls-in-oracle.html

Sunday, September 16, 2012

Quick Tip - Oracle SQL*Plus: HTML output

Recently, I posted about SQL*Plus, a database querying tool from Oracle. Like I mentioned there, the tool is rudimentary and has some basic file handling capabilities. It has a command, SPOOL, to write the console output to a file. Traditionally, you only spooled the output to a text file. Some releases ago (I believe in 8i), Oracle included option to spool the output in HTML format. Useful when you are dumping data from tables (Have you tried dumping table data in plain text file format??) and for generating simple reports.

To generate HTML output from SQL*Plus, all you have to do is add markup option either on command line or inside the script. By the nature of it, it is only useful when you are spooling the output. Just add below to the script before spooling: and the output will be spooled to the file mentioned  in HTML format.
SET MARKUP HTM ON SPOOL ON PREFORMAT OFF ENTMAP ON -
HEAD "<TITLE>My TABLE Listing
</TITLE> -
<STYLE type='text/css'> -<!-- BODY {background: #FFFFFF} --> -
</STYLE>" -
BODY "TEXT='#000FF'" -
TABLE "WIDTH='90%' BORDER='5'"


...
spool dump_table.html


...

SPOOL off

That's it. Just be aware that the output file will be larger than the usual text file spool. By the way, the hyphen "-" at the end of each line is the line continuation character in SQL*Plus. So, it's really one line command with several options.

Alternatively, you can specify the HTML option on the commandline as shown below:
sqlplus -S -M "HTML ON" <user>/<password>@<SID> @dump_table.sql > dump_table.html

The option -M "HTML ON" is equivalent to MARKUP HTML ON in the script above. The option, -S is for silent, which turns off spurious output from SQL*Plus.

 

See here for more about creating HTML reports using SQL*Plus.

Friday, December 9, 2011

Forward Slash in SQL*Plus

Note to the visitor: Thank you for stopping by. I see a lot of people trying to find out about forward slash in SQL*Plus. I want to clarify: This post only applies to Oracle SQL. If you are you are querying about a "/" usage in any other database, you are in the wrong place.

Forward Slash ("/") in Oracle SQL

This post is about using forward slash ("/") in Oracle SQL and PL/SQL. This gets a lot of developers (new and experienced alike) as it's not always needed and sometimes it shouldn't be used.

Have you ever tried typing "/" (without quotes), as soon as you logged into SQL*Plus? Try it. You will see something like below.
SQL> /
SP2-0103: Nothing in SQL buffer to run.
SQL>

The above snippet explains it all pretty well. Forward slash ("/" ) is a shortcut for RUN command in SQL*Plus (like "go" in SqlServer or mysql). Since there is nothing to run yet, SQL*Plus returned an error message (prefixed with SP2-).

The message in the above example also shows there is a SQL buffer in SQL*Plus. Any SQL sent to DB ends up in this (one statement) buffer. Also, if there is a statement in the buffer, "/" would have simply re-executed it.

When you type a line and press ENTER, SQL*Plus checks if it's one of it's own commands, a SQL statement or just bad text. If it's a SQL (identified by keywords like SELECT etc), it starts storing it in the buffer until it sees a "/" or a semi-colon, at which time it executes it (actually sends it to the server to execute it).

Gotcha with Forward Slash (/)

SQL Statements typically end in Semi-colon. Semi-colon is the Statement Terminator in ANSI standard. But, in Oracle SQL, you can use "/" to run a SQL, instead of semicolon, like shown. (This is how SPANISH record got inserted in my attached example at the bottom of this post).
SQL> SELECT dummy FROM dual
2 /
Dummy
-----
X

But, here is the gotcha: If the statement itself contained a semi-colon, then "/" would have re-executed it!!!!
SQL> select dummy from dual;
Dummy
-----
X
SQL> /
DUMMY
----------
X

Oops. Imagine this was an INSERT statement instead!!! (And this is what happened in my example script attached. If you notice the "/" before commit, this is what caused FRENCH to be inserted twice!!).

So be careful with your usage of "/". For plain SQL, it's a matter of choice. Typically, this is used in DDL statements and semi-colons in DML statements. Be consistent to avoid surprises.

Remember "/" is just a RUN (or push) command in the tool, not part of Oracle SQL itself. If you run my attached example script in SQLTools, you will get an error on "/".

Always, run your scripts in SQL*Plus to make sure it will run fine in production, as this is a tool of choice for several DBAs.

Forward Slash ("/") in PL/SQL

PL/SQL on the other hand is a group of SQL statements with embedded semicolons. In this case, a semicolon alone cannot be used to send to DB. It needs a push with "/"! So, in this case, it acts as a Statement Terminator as well.
SQL> list
1 DECLARE
2 x NUMBER;
3 BEGIN
4 SELECT 10 as num INTO x FROM dual;
5 Dbms_Output.put_line('x = ' || x);
6
7 END;
8
9
10
11
12
13*
SQL>

In the above example, Notice lines  7 - 13 are blank. This is because I pressed ENTER several times. Since SQL*Plus knows this a PL/SQL (declare, begin...end), it waits for me to signal the real end, which is..... a "/". Until I typed "/" on line 13, it didn't come out of the PL/SQL editing mode. "/" also pushed the PL/SQL block to server to execute it.

Even though this has a bunch of lines, this whole text is a single PL/SQL statement. Type list at the SQL*Plus Prompt. You will see the "single" statement listed in full (minus the "/").

SQL*Plus commands

SQL*Plus commands like SET, SHOW, SPOOL etc are typically one liners and should not be ended with semicolon (and don't need "/" either).
SQL> show serveroutput;
serveroutput OFF
SQL>

These commands will be executed locally in the tool and won't be stored in SQL buffer. So, a "/" after a SQL*Plus command will not re-execute it. In fact, it will re-execute the previous SQL Statement in the buffer!!

(Note: A "/" must be the first non-whitespace character (space, tab etc) on a line by itself. It cannot be at the end of a line, like semi-colon can be).

I've also posted a more detailed description of SQL*Plus here.

Notes

  1. There is a subtle difference: RUN actually lists before it runs; / doesn't!

  2. This is a local message from SQL*Plus. If this was from Oracle database server, you would see a "ORA" prefix.


Example Script
-- Example data for SQL*Plus related blogs
DROP TABLE greetings;
CREATE TABLE greetings (id NUMBER, lang VARCHAR2(10), msg VARCHAR2(30));
INSERT INTO greetings VALUES (1, 'ENGLISH', 'Hello World');
INSERT INTO greetings VALUES (2, 'FRENCH', 'Bonjour le monde');
/
INSERT INTO greetings VALUES (3, 'GERMAN', 'Hallo Welt');
list
/
INSERT INTO greetings VALUES (4, 'SPANISH', 'Hola Mundo')
/
COMMIT;
-- Here are some extra notes about this script:
-- Just Hello world in some languages. Translations are approx. from Google.
-- Even though we are submitting this script to SQL*Plus entirely,
--   it executes one SQL Statement at a time. List command shows this.

-- Forward Slash after FRENCH, reexecutes previous command. so you will see
-- 2 FRENCH records inserted.
-- Forward Slash after GERMAN was, by mistake, thought to run list, but instead it
-- it ran the previous SQL statement, thus we have 2 GERMANs!!
-- Forward Slash after SPANISH inserts only once - because it's missing semi-colon (;)
-- and / substitutes for it!!
-- Commit is required in Oracle SQL. By default, Autocommit is not turned on in SQL*Plus
-- To see/set default use SHOW/SET AUTOCOMMIT

Further Reading



  • See my Take 2 on this topic here.

  • If you are looking for any relationship between forward slash and commits (there is none), please see my post here.

  • If you are looking to see if Forward Slash somehow affects DDLs, please see here.

  •  Please see here for a more detailed description of SQL*Plus in general.


References

SQL BNF Grammars

  1. http://savage.net.au/SQL/sql-92.bnf.html

  2. http://www.contrib.andrew.cmu.edu/~shadow/sql/sql2bnf.aug92.txt

  3. http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt

Oracle SQL Gotcha - Commit in DDL

This happened to one of my fellow developers recently. She had an ALTER statement in the middle of a SQL script that produced an unexpected results when the SQL failed. The problem was mixing DMLs with DDLs in the same script. Read on!

As you probably know, Commits work differently in DMLs and DDLs in Oracle.

  • Typically* DML statements (like INSERT, UPDATE, MERGE, DELETE...) needs an explicit COMMIT statement to commit changes to DB.



  • DDL statements (like CREATE, CREATE TABLE AS (CTAS), DROP, TRUNCATE etc) don't require a COMMIT statement. DDL actually issues an implicit COMMIT before and after the statement is executed.  This also means, any COMMIT statement after a DDL is not required and has no effect.


So, if you are mixing DMLs and DDLs, be mindful of this. If you have a DML (insert/update...) statement, followed by a DDL statement, you might have inadvertently committed earlier DML. Any later rollbacks won't have any effect.

Here is an example. Test 1 doesn't mix DMLs and DDLs. (There is no DDL immediately after DML). Test 2 on the other hand has a bug in that it has a DDL after the DML statement which results in unpredictable results.

The gotcha here is that, since the commit is done implicitly before even the DDL is executed, this happens even if the DDL failed!! 

Test 1 -- to show the effect of rollback after a DML
-- 1. DDLs 
DROP TABLE test_commit;
CREATE TABLE test_commit (id int, name VARCHAR2(30));

-- 2. DML
INSERT INTO test_commit VALUES(1, 'Test 1');
-- this shows 1 row
SELECT * FROM test_commit;

-- 3. Rollback below reverses the inserts (no DDLs since last DML)
ROLLBACK;

-- 4. This shows no rows
SELECT * FROM test_commit;

Test 2 - show the effect of an interspersed DDL on commit!
-- 1. DDLs
DROP TABLE test_commit;
CREATE TABLE test_commit (id int, name VARCHAR2(30));

-- 2. DML
INSERT INTO test_commit VALUES(1, 'Test 1');
-- This shows 1 row
SELECT * FROM test_commit;

-- 2a. This DDL implicitly commits, so above insert is now committed.
DROP TABLE temp_table2;

-- temp_table2 doesn't really exist - I just used it to show it really doesn't matter,
-- commit happens implicitly in the DDL

-- 3. This rollback is same as in Example 1, but this has no effect
ROLLBACK;

-- 4. his shows 1 row; if the rollback above worked, we wouldn't see any rows here.
SELECT * FROM test_commit;

-- End of Test; cleanup
DROP TABLE test_commit;

Notes:
* When Autocommit is off, which is the default in Oracle

Tuesday, October 25, 2011

Oracle SQL*Plus

Oracle SQL*Plus


I've posted a few SQL tricks earlier. Most of these were in Oracle and often times, we use SQL*Plus to run these.There are several sophisticated SQL clients available for Windows (Toad is a great tool and at work we use SQL Tools++ and Golden), many even free, but SQL*Plus still remains indispensable. In this post, I will explore this tool.

SQL*Plus is an interesting tool. It's an old, text based, SQL Client that has been in Oracle from the beginning. Whatever it's strengths, user interface isn't one of them. With a command line interface (there is a Windows version (SQLPLusw), but not a major improvement) and quirky commands, it is not exactly user friendly, at least not according to today's GUI standards. Would you believe that SQL*Plus is supposed to be an improvement over what Oracle called "User Friendly Interface"?

Oracle did try to come up with several tools to replace it- a browser version called iSQL*Plus, a java based tool called SQL*Developer, but SQL*Plus survived and may actually outlive those. (See here for the bad news for  iSQL*Plus and SQLPlusw in 11g. Among other things, they actually recommend SQL*Plus!!!).  See Lauren's blog for the official links about this announcement. Also don't forget to read up this blog setting up SQL*Plus nicely in windows, so you won't miss those little used tools.

Strengths of SQL*Plus

Here are some of the reasons why SQL*Plus is still used in production environments, in spite of it's limitations:

  • For one, it is a very stable product. This combined with it's simplicity and small memory footprint, makes it a tool of choice for production.

  • It is available on various platforms, and offers an almost identical user interface across different platforms.

  • The tool can be run in batch mode; combining this with spooling capability and some commands, we can automate running scripts.

  • The tool conforms to the Oracle SQL standards so closely, so it still remains as tool of choice when you want to ensure your SQL will really pass in the Oracle environment.

  • And to it's credit, it has added few features over the years. For e.g., you can actually spool it's output to an HTML file by adding a few directives (see my post here).


SQL*Plus as a Client/Server Tool


SQL*Plus, like any other SQL Client, is an interactive Client/Server tool. It has a simple editor (don't expect a fancy editor - the one built in is a rudimentary line editor), which can hold 1 statement (however long) in the buffer. You can open a session in SQL*Plus and start typing Statements at the prompt. After you type a line, you can press enter to go to the next line. SQL*Plus will show a line # to indicate it's ready for the next line. You can keep typing and when you are ready, you can press Enter twice, for it to leave edit mode. At this point, the Statement is in memory ready to be executed. When you are done entering/editing the contents, simply issue a “RUN” command (See below) and the contents (SQL) will be pushed to the DB (server) to execute and return results. When the statement is executed successfully, the statement is then put into the 1-statement buffer for reuse.

Below is my sincere effort to document how I see SQL*Plus is executing commands. It's only an educated guess, so use with caution. SQL*Plus by design is a simple tool that executes one command at a time.

[caption id="attachment_471" align="alignnone" width="984"] Fig 1. SQL*Plus Design - an educated guess[/caption]

 

As shown there are 3 different types of statements that are processed by the command processor. It uses the keyword and the end of the statement to decide where to send it. If the statement has a semi-colon (";") or a slash ("/") at the end (See below), it is sent to the Database engine. (Try typing anything with a keyword (like SELECT etc) and end it with a semi-colon, you will get a SQL Error, indicating it went to DB Engine). Rest of them are tried as a local tool command.

Image 744

 

 

 

Image 746


 

SQL*Plus as a batch tool

There also seems to be a loop in the command processor, that could read one command at a time from a file and pass it on to the command processor. This combined with few commands to SET and SPOOL makes it a good batch tool. A SQL file is just a combination of different types of statements (SQL, PL/SQL and tools commands).

To run a file with SQL statements you can use START command. The short-cut for this is @. So, run a SQL file, you simply have to type,

@<sqlfile>, where <sqlfile>is any text file with SQL statements.

You can mix tool commands and SQLs in the file. The thing to remember is when we execute a SQL file in SQL*Plus, it processes one statement at a time successively. This is key to understand any problem with your script. You are not passing the SQL file to Oracle, you are giving it to SQL*Plus, which reads and executes one statement at a time depending on the type of statement and terminators used.

Run command and Statement Separator


Oracle uses it's own Run command "(/") and SQL standard for Statement Terminator to execute SQLs.

  • Oracle uses Run command as a "statement pusher" to send to DB. This has a short-cut ("/").

  • Oracle also allows semicolon (SQL Standard) to terminate SQLs.

  • As yet another short-cut, SQL*Plus pushes SQL statements to DB right after a semicolon, so you could use either slash or semicolon there.

  • But, with PL/SQL you needed both, as the SQL block will need a semi-colon and to push it to DB, you need a "/".


Potential issues with Run Command

This flexibility in SQL*Plus typically leads to many errors in SQL Scripts. Particularly,

  • If you have a PL/SQL in a file and missed the "/", it won't execute the PL/SQL.

    • Imagine, you try to drop a table and recreate it with few changes. If the drop command failed, results of successive commands may become unpredictable



  • On the other hand, if you added a "/" after a SQL with semicolon, the Statement is run twice!!

    • For e.g., you could end up in inserting a record twice which could make some other Select/into to fail downstream.




(Different databases handle this issue differently. tools for SQL Server for e.g., uses "go" as "statement pusher" uniformly for all statements. If you miss a "go" you get an error. MySql handles this slightly differently It uses semicolon for simple SQLs and when it comes to block SQL, it allows us to define a temporary delimiter other than semicolon to mark end of SQL).

See my post here for more details on "/".

How to avoid issues


We can use standards to avoid surprises. I prefer to use semicolons after all SQL (DML and DDL) statements. "/" for PL/SQLs. You may find "/" used for DDLs as well.

If you are planning to use "/" after a SQL statement in a file, I suggest this only for the last SQL statement in the file. This is just a preference for readability.

Few related SQL*Plus gotchas

1. Though both semicolon and "/" could push a SQL statement to Oracle, there is a subtle difference. you can type multiple statements in 1 line separated by semicolon. "/" cannot be used thus.

2. The "/" must be the only character on the line by itself. No other character except spaces may be allowed, not even comments.

3. Sometimes inline comments that start with /* could be construed as "RUN" and may cause syntax errors.

4. Typically, you can include blank lines inside PL/SQL block. But SQL*Plus may frown if you had blank lines inside simple SQL statement. There is SQL*Plus setting (sqlblanklines) to avoid this, but I suggest not to use it to stick to standards.

Reference

As mentioned, SQL*Plus can execute SQL and PL/SQL statements and several of it's own commands. Oracle supports Standard SQL and has several extensions as well. Oracle has very good documentation on this topic and several others. See here for Oracle SQL reference and here for good reference on PL/SQL. See here for a complete reference on SQL*Plus from Oracle. refer to this site for the SQLPlus tool commands.

Links

http://www.williamrobertson.net/documents/sqlplus-setup.html

http://thomas.eibner.dk/oracle/sqlplus/

http://www.adp-gmbh.ch/ora/sqlplus/

http://mti.ugm.ac.id/~adji/courses/resources/CICC/Short%20Cicc/text/or8_text/ch2.htm

http://download.oracle.com/docs/cd/B14117_01/server.101/b12170/ch2.htm

http://ugweb.cs.ualberta.ca/~c391/manual/chapt4.html

http://laurentschneider.com/wordpress/2006/03/isqlplus-and-sqlplusw-desupport.html