tag:blogger.com,1999:blog-7853538905902841142023-11-15T09:33:27.574-08:00Power to Build (Software)Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.comBlogger83125tag:blogger.com,1999:blog-785353890590284114.post-29537910783298840222013-06-19T09:31:00.000-07:002013-07-04T07:53:49.114-07:00Quick Tip: Oracle Sqlplus - changing prompt<div dir="ltr" style="text-align: left;" trbidi="on">
Oracle SQL*Plus has several settings that can be customized using SET command. When you run SQL*Plus, typically the following (boring?) prompt greets you.<br />
<br />
<strong>SQL></strong><br />
<br />
If you ever want to change this to something more interesting, there is a SET option for that.<br />
<br />
<blockquote class="tr_bq">
SET <strong>sqlprompt </strong>'sqlplus:&_user@&_connect_identifier > '</blockquote>
<br />
This will be shown as,<br />
<br />
<blockquote class="tr_bq">
sqlplus:<user>@<SID>></blockquote>
<br />
For e.g.,<br />
<br />
<blockquote class="tr_bq">
sqlplus:scott@hr></blockquote>
<br />
Where scott is a user id logged into hr database.<br />
<br />
<strong>References</strong><br />
<br />
<a href="http://docs.oracle.com/cd/B28359_01/server.111/b31189/ch12040.htm" target="_blank">http://docs.oracle.com/cd/B28359_01/server.111/b31189/ch12040.htm</a><br />
<a href="http://docs.oracle.com/cd/B19306_01/server.102/b14357/ch12017.htm#i2675128" target="_blank">http://docs.oracle.com/cd/B19306_01/server.102/b14357/ch12017.htm#i2675128</a></div>
Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-35310904902244172742013-06-19T09:13:00.000-07:002013-06-24T08:20:23.710-07:00Java Stored ProceduresI am sure you are familiar with Stored Procedures. Almost every major database vendor supports it. They are program units that are precompiled and stored inside the database. Since the program is inside the database, it is tightly coupled with the database SQL language constructs. This and the precompiled nature of these programs, make the database access faster. So, even if you are writing external programs that access the database, you will benefit from coding and combining the database operations in stored procedures. This saves a lot of back and forth and thus network traffic. Since version 7, Oracle has support for Stored Procedures. These programs are coded a in native language called PL/SQL.<br />
<div dir="ltr">
Since Oracle 8i, Oracle supports coding and running similar procedures (program units) written in Java. Like PL/SQL procedures, these programs are precompiled and stored inside the database and are known as Java Stored Procedures. The JAVASPs(1) are stored as classes (in blob fields) and when invoked they are run inside a JVM that runs within Oracle database. Prior to Oracle 10g, they needed to be wrapped inside a PL/SQL procedure or package. Since Oracle 10g, you can actually invoke Java classes directly from SQL (just the same way you call a PL/SQL function in a SQL).</div>
<br />
JavaSPs are different from regular java programs in that they actually run inside a VM within the database. There are a few restrictions while coding a JAVASP.<br />
<h3>
Building and Deploying Java SP</h3>
<br />
<div dir="ltr">
To write and build the Java stored procedures, you can use your standard Java development environment. I use eclipse to develop and Ant to build it</div>
<br />
<div dir="ltr">
<a href="http://power2build.files.wordpress.com/2013/06/ant_build.png"><img alt="" class="alignnone size-full wp-image-5063" height="159" src="http://power2build.files.wordpress.com/2013/06/ant_build.png" width="511" /></a></div>
<br />
build_LATAXSP.xml has steps to compile java classes and build jar file.<br />
<h3>
Deploying Java SP</h3>
<br />
At this point the jar file is ready to be deployed to Oracle. Typically we pass this step onto the DBA who will then load the jar into Oracle Database instance. But during development, developer can load these themselves using the loadjava utility. This is typically available on the machine where the database is running. (Remember, it is run by DBAs?). In our case we have the Oracle databases running on Unix boxes, so we have loadjava utility available there. I upload the file to Unix and run loadjava. While uploading make sure it’s in binary mode.<br />
<div dir="ltr">
Below screenshot shows a sample run of loadjava command on Unix.</div>
<br />
<a href="http://power2build.files.wordpress.com/2013/06/loadjava_results2.png" target="_blank"><img alt="loadjava_results" class="alignnone wp-image-5068" height="583" src="http://power2build.files.wordpress.com/2013/06/loadjava_results2.png" width="853" /></a><br />
<br />
In this example, I loaded all the classes in a jar file, to the database. As shown there were 24 classes and 2 resources loaded and there were no errors. If the command failed to load the Java classes, you will see an error message here. The first time around, all the classes files are loaded. Next successive load of the same jar file, will load only classes that have been modified since the last load.<br />
<br />
<strong>Verifying the load</strong><br />
<div dir="ltr">
To view the objects in Oracle, following SQLs can be used:</div>
<br />
select * from all_objects where object_type = 'JAVA CLASS' and owner = <owner>;<br />
<div dir="ltr">
or</div>
<br />
<div dir="ltr">
select * from user_objects where object_type = 'JAVA CLASS';</div>
<br />
To see a little more detail about the Java stored procs, use<br />
<div dir="ltr">
SELECT * FROM user_java_classes; -- this lists java procs for the user.</div>
<br />
<div dir="ltr">
Earlier I posted about displaying contents of the resources files (text files) loaded above.</div>
<br />
<strong>Notes:</strong><br />
(1) Oracle actually refers to Java Stored Procedure as JSP. To avoid confusion with Java Server Pages, I prefer JavaSP.<br />
<br />
[gallery include="5064,5065"]Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-17882817151953810112013-06-18T03:49:00.000-07:002013-06-20T22:01:47.201-07:00Gotcha: Accessing Oracle 11g with older SQL*PlusOracle 11g has several<span style="text-decoration:underline;"><strong> <a href="http://www.oracle.com/technetwork/articles/sql/11g-misc-091388.html" target="_blank">enhancements </a></strong></span>over the previous versions. Some of these changes are in the client tools like SQL*Plus. To be able to use these you actually need to upgrade the client tools as well. If you are using old clients like me (I use Oracle 10g SQL*Plus), then you won't see those changes in Oracle 11g. For e.g., the EXITCOMMIT option I mentioned in my earlier<span style="text-decoration:underline;"><strong> <a title="Quick Tip: Oracle – Commit on Exit" href="http://power2build.wordpress.com/2013/06/18/quick-tip-oracle-commit-on-exit/" target="_blank">post </a></strong></span>cannot be used in my scripts!<br/><br/>(I couldn't upgrade my client because we use Powerbuilder 10.2 with EA Server 5.5 which doesn't support Oracle 11g drivers. So, until we upgrade those tools, we are stuck with 10g client!)Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-24756489520834640882013-06-18T03:35:00.000-07:002013-06-20T22:01:47.204-07:00Quick Tip: Oracle - Commit on ExitOracle by default does not automatically commit a transaction (Autocommit is typically off). Transaction Management (Commit or Rollback) is left to the client as this <a href="http://asktom.oracle.com/pls/apex/f?p=100:11:0::NO::P11_QUESTION_ID:314816776423" target="_blank"><span style="text-decoration:underline;"><strong>askTom post </strong></span></a>explains it nicely. The tool you use may provide this option for you. For e.g., SQL*Plus commits any open transaction upon exit. So, even if you don't have COMMIT at the end of a script, when SQL*Plus will issue a COMMIT when the SQL is exited.<br/><br/>This was the default behavior for a long time and was generally accepted. Some developers wanted to have more control over it, and according to this <span style="text-decoration:underline;"><strong><a href="http://stackoverflow.com/questions/1368092/why-does-sqlplus-commit-on-exit" target="_blank">stackoverflow post</a></strong></span> there was even a bug (633247) opened for it in 1998! Recently, in Oracle 11g this has finally been changed. Now, as of Oracle 11g R2, the user actually has an option (<span style="text-decoration:underline;"><strong><a href="http://www.acehints.com/2011/07/oracle-11g-r2-sqlplus-set-exitcommit.html" target="_blank">EXITCOMMIT</a></strong></span>) to tell SQL*Plus whether to COMMIT or ROLLBACK upon EXIT. Following statement makes SQL*Plus roll back a transaction upon exit.<br/><pre>set exitcommit off</pre><br/>See this <span style="text-decoration:underline;"><strong><a href="http://www.oracle.com/technetwork/articles/sql/11g-misc-091388.html" target="_blank">O</a></strong></span><b><span style="text-decoration:underline;"><strong><a href="http://www.oracle.com/technetwork/articles/sql/11g-misc-091388.html" target="_blank">racle Post</a></strong></span></b> for a good example.<br/><br/><strong>Notes:</strong><br/><ul><br/> <li>One poster in the Stackoverflow post above, actually mentions about Autocommit option in SQL*Plus. I just want to clarify:</li><br/></ul><br/>SQL*Plus does have an option to set Autocommit on or off. This is actually meant for every statement that you issue, not just the last statement before exit. Prior to Oracle 11gR2, this did not have an impact on the Commit on Exit!<br/><ul><br/> <li>Also, this change is actually in client tool (SQL*Plus) not in the database. So, if you are using an older client (like I am), you will be disappointed not to find this option!</li><br/></ul><br/><strong>References</strong><br/><br/><a href="http://stackoverflow.com/questions/1368092/why-does-sqlplus-commit-on-exit">http://stackoverflow.com/questions/1368092/why-does-sqlplus-commit-on-exit</a><br/><br/><a href="http://www.oracle.com/technetwork/articles/sql/11g-misc-091388.html">http://www.oracle.com/technetwork/articles/sql/11g-misc-091388.html</a><br/><br/><a href="http://www.acehints.com/2011/07/oracle-11g-r2-sqlplus-set-exitcommit.html">http://www.acehints.com/2011/07/oracle-11g-r2-sqlplus-set-exitcommit.html</a><br/><br/><a href="http://asktom.oracle.com/pls/apex/f?p=100:11:0::NO::P11_QUESTION_ID:314816776423">http://asktom.oracle.com/pls/apex/f?p=100:11:0::NO::P11_QUESTION_ID:314816776423</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com1tag:blogger.com,1999:blog-785353890590284114.post-36229219550854964352013-06-13T03:47:00.000-07:002013-06-20T22:01:47.209-07:00Gotcha: PB Deployment - SYSTEM var not supported errorWhen you work on code deployed to n-tier, you have to be little more careful while defining the interface for the objects. An n-tier application typically runs in an heterogeneous environment, meaning objects in different languages/platforms may talk to each other through a common interface. If you are not careful, your object may end up being passed to the wrong tier that doesn't know how to interpret it. You have to make sure the data types of the interface functions exist on all sides. Otherwise, you will get unexpected errors at compile or run time.<br/><br/>This is true for PB also, if you are deploying your code to EA Server. Mixing system objects and types in the object's interface will cause errors while deploying to EA Server as shown below.<br/><br/>[caption id="attachment_5001" align="aligncenter" width="536"]<a href="http://power2build.files.wordpress.com/2013/06/corba_idl_error_on_pb_sysvars1.png"><img class="wp-image-5001 " alt="1. CORBA IDL Error on PB System objects" src="http://power2build.files.wordpress.com/2013/06/corba_idl_error_on_pb_sysvars1.png" width="536" height="292" /></a> 1. CORBA IDL Error on PB System objects[/caption]<br/><br/>While deploying this project (p_d_adm_roles_lh) to EA Server, PB throws the following error message:<br/><pre><em><code><code><br/> ---------- Deploy: Deploy of p_d_its_adm_roles_lh<br/>Doing Incremental Rebuild...<br/>Generating IDL for Selected Components...<br/>Generation Messages:<br/> Deployment Warning: SYSTEM Variables Not Supported. The following bypassed for component 'im_adm_roles_impl': n_txn<br/> Deployment Warning: SYSTEM Variables Not Supported. The following bypassed for component 'im_adm_roles_impl': n_txn<br/> ---------- Finished Deploy of p_d_its_adm_roles_lh<br/></code></code></em></pre><br/>Here is what <a title="PB Deployment Errors" href="http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.eas_5.2.eastg/html/eastg/CDDDGIFJ.htm" target="_blank">Sybase document</a> says:<br/><div><br/><br/><b>Deployment error or warning (from PowerBuilder): SYSTEM variables not supported</b><br/><blockquote>Public instance variables and arguments to public functions can be any of: Standard datatypes, Structures, Custom class user objects that have been deployed as EAServer components, ResultSets.</blockquote><br/></div><br/><blockquote><br/><div>If you are using <b>system datatypes</b> (transaction, data store, and so on) as instance variables, declare them as protected or private. If you are using system datatypes as function arguments, declare the function as protected or private.</div><br/><div></div></blockquote><br/><div>The real problem was in the CORBA interface. I declared a public instance variable called n_txn of type <strong><a title="Transaction" href="http://powerbuilder.hyderabad-colleges.com/Introduction-to-PowerBuilder-1-9-20.html" target="_blank">Transaction</a> </strong>which is known to PB but not to <strong><a href="http://www.slideshare.net/arnoldindia/corba-idl" target="_blank">CORBA IDL</a></strong> (interface definition language). So, this cannot be in the public interface of the object. CORBA IDL only allows types that can be mapped to it's own. User defined objects are allowed, as long as they themselves are composed of simple types that IDL allows. More on these in a separate post.</div><br/><div></div><br/><div></div><br/><div>While deploying a component to EA Server, PB generates IDL and related proxy and stubs/skeletons for the CORBA interface. It checks to make sure generated IDL complies with the rules of IDL definitions. In the above example since PB's Transaction object cannot be mapped to any IDL type, PB prevents deployment and throws the error. Thus avoiding surprises at runtime.</div><br/><div></div><br/><div>Ideal solution is to avoid non-standard types in the instance/shared variables in objects to be deployed to n-tier. But, if you really have to define instance/shared variables of types that are not translatable to IDL, then make them private or protected. IDL interface is generated only for the public attributes and methods.</div><br/><div></div><br/><strong>References:</strong><br/><a href="http://www.google.com/url?q=http%3A%2F%2Finfocenter.sybase.com%2Fhelp%2Findex.jsp%3Ftopic%3D%2Fcom.sybase.help.eas_5.2.eastg%2Fhtml%2Feastg%2FCDDDGIFJ.htm&sa=D&sntz=1&usg=AFrqEzdnRmTFqUmU4Tw_akjvAyUW-Z48_A" rel="nofollow">http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.eas_5.2.eastg/html/eastg/CDDDGIFJ.htm</a><br/><br/><a href="http://www.omg.org/gettingstarted/omg_idl.htm#TypeSystem">http://www.omg.org/gettingstarted/omg_idl.htm#TypeSystem</a><br/><a href="http://www.ibm.com/developerworks/webservices/library/co-corbajct3.html">http://www.ibm.com/developerworks/webservices/library/co-corbajct3.html</a><br/><a href="http://documentation.progress.com/output/Iona/orbix/gen3/33/html/orbix33java_pguide/IDL.html">http://documentation.progress.com/output/Iona/orbix/gen3/33/html/orbix33java_pguide/IDL.html</a><br/><a href="http://www-cdfonline.fnal.gov/daq/CORBAXXX/tutorial.html">http://www-cdfonline.fnal.gov/daq/CORBAXXX/tutorial.html</a><br/><a href="http://docs.oracle.com/cd/E13203_01/tuxedo/tux80/cref/member.htm#1120423">http://docs.oracle.com/cd/E13203_01/tuxedo/tux80/cref/member.htm#1120423</a><br/><a href="http://www-itec.uni-klu.ac.at/~harald/corba/idldatattypes.html">http://www-itec.uni-klu.ac.at/~harald/corba/idldatattypes.html</a><br/><a href="http://www.slideshare.net/arnoldindia/corba-idl">http://www.slideshare.net/arnoldindia/corba-idl</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-49205637300855407112013-06-12T07:10:00.000-07:002013-07-04T07:56:27.646-07:00Oracle: Commit and Forward Slash<div dir="ltr" style="text-align: left;" trbidi="on">
Further to my post earlier about <a href="http://power2build.wordpress.com/2013/06/12/oracle-ddl-and-forward-slash/" target="_blank" title="Oracle: DDL and Forward Slash">DDL and Forward Slash</a>, I would like to explain Forward slash's role (or lack thereof) in other scenarios. One such scenario people seem to get confused with is COMMIT. They seem to want to find some type of connection between the two. Let me say it upfront, there is no connection between COMMIT and "/".<br />
<br />
Forward Slash is simply a SQL Executor and it happens to show up in various situations. This combined with SQL*Plus's own quirks seem to imply certain other functions for this character. Let me repeat: it's just an executor - short for RUN command in SQL*Plus.<br />
<br />
One reason it could be construed as connected with COMMIT is that it may often be the last character in a file. When you run SQL*Plus in batch mode (running a script in a file), it typically exits after the last line (on *nix. It doesn't DoS/Windows). And the SQL*Plus's own behavior is to COMMIT a transaction when exiting. (Why? See <strong><a href="http://stackoverflow.com/questions/1368092/why-does-sqlplus-commit-on-exit" target="_blank">here</a></strong> for a nice discussion about this.) So, it's the exit that committed not the "/". But. you can see why some newcomer to Oracle who just inherited some scripts, may think that "/" (being the last statement) actually committed the transaction!!!<br />
<br />
The other reason could be DDL. DDLs are implicitly committed. (Actually it commits before and after the DDL statement itself, so beware if you are mixing DMLs and DDLs).<br />
<br />
Now, typically a DDL could be ended with "/". For e.g.,<br />
<pre class="brush:sql">
INSERT INTO department(100, 'HR');<br />
CREATE TABLE employee(<br />
emp_id NUMBER,<br />
emp_name VARCHAR2(40)<br />
/</pre>
<br />
In the above case, department table will have the entry, even if the CREATE TABLE below failed. In generla, any DML (like INSERT/UPDATE/DELETE) before a DDL, DML would have been committed, even if the DDL itself failed. Since "/" is the last statement here, one could think, "/" did it!!! Another strike against "/".<br />
<br />
So, trust me folks! "/" doesn't do anything else, except sending a SQL to Oracle in SQL*Plus. It's just a shortcut for Run Command. But, depending on the situation, it may look like it's doing something more. In such cases, analyze the SQL and try to rearrange them. Try to keep DMLs separate from DDLs to avoid surprises. And of course, set some standards for punctuations and stick to it to avoid any undesired effects.</div>
Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com1tag:blogger.com,1999:blog-785353890590284114.post-87058113625511854242013-06-12T03:02:00.000-07:002013-06-20T22:01:47.225-07:00Oracle: DDL and Forward SlashIn this blog, I write about various technologies I come across. Only a few topics seem to get a lot of attention. One of them happens to be Forward slash in (Oracle) SQL. I've already posted about this <strong><a title="Forward Slash in SQL*Plus" href="http://power2build.wordpress.com/2011/12/09/forward-slash-in-sqlplus/">here</a> </strong>and <strong><a title="Oracle: Forward Slash in SQL*Plus – Take 2" href="http://power2build.wordpress.com/2013/06/05/oracle-forward-slash-take-2/">here</a>. </strong>When I look at the search terms that lead to my site,<strong> </strong>I sense a lack of understanding of it's usage. A search through the web quickly reveals that forward slash has been a source of confusion even among some of the experienced Oracle SQL developers <strong><sup>1</sup></strong>. Based on the search terms, I can see the newcomers are often getting confused by the SQLs left for them by their predecessors and are looking for a clear explanation. "/" is often mixed in various statements, such as DDL, COMMIT etc and this leads to confusion about the statement itself. I will try to explain the role of "/" (or lack thereof) in each different scenario in separate posts.<br/><br/>One of the search term that I get a lot is "DDL and Forward slash" or some variation of it. Forward slash has nothing to do with DDL per se. If you saw my other posts, it is just a statement executor (RUN command for both SQL and PL/SQL). Unfortunately, semi-colon doubles as statement terminator and executor for SQLs (poor choice of shortcut) in SQL*Plus. There lies the confusion. Thus, the CREATE TABLE or DROP VIEW statements can be terminated by a semi-colon as well as "/".<br/><br/>For e.g.,<br/><pre>CREATE TABLE employee(<br/> emp_id NUMBER,<br/> emp_name VARCHAR2(40));</pre><br/>and<br/><pre>CREATE TABLE employee(<br/> emp_id NUMBER,<br/> emp_name VARCHAR2(40))<br/>/</pre><br/>are identical as for SQL*Plus is concerned. But, as a matter of preference, people tend to use "/" in DDL.<br/><br/>But this is not the case in other DDL statements that involve PL/SQL:<br/><pre>CREATE OR REPLACE FUNCTION f_get_employee(a_emp_id NUMBER)<br/>RETURN VARCHAR2<br/>AS<br/> v_emp_name VARCHAR2(40);<br/>BEGIN<br/> SELECT emp_name<br/> INTO v_emp_name<br/> FROM employee<br/> WHERE emp_id = a_emp_id;<br/>END;<br/>/</pre><br/>Here "/" is a must, as the above statement is one PL/SQL statement (from CREATE to END;). Remember, PL/SQL block ends with END; and it has to be followed by "/" to be executed <strong><sup>2 (see notes below)</sup></strong>. Thus <span style="color:#ff0000;"><strong>some</strong> </span>DDL statements (those that create programming constructs like packages, procedures, triggers) require "/" at the end. This, I think, led the newcomers to believe there is a link between "/" and DDL!!! There is <strong>no</strong> connection! You cannot generalize the use of "/" for all DDLs. For e.g., if you tried to do this (semi-colon followed by "/") in a CREATE TABLE statement, you will have unexpected behavior:<br/><div style="background-color:black;color:white;">SQL> CREATE TABLE employee(<br/>2 emp_id NUMBER,<br/>3 emp_name VARCHAR2(40))<span style="text-decoration:underline;"><strong>;</strong></span><br/>Table created.SQL><span style="text-decoration:underline;"><span style="color:#ffffff;"><strong> /</strong></span></span><br/>CREATE TABLE employee(<br/>*<br/>ERROR at line 1:<br/>ORA-00955: name is already used by an existing objectSQL></div><br/>In this case, the semi-colon followed by slash is a bad idea.<br/><br/>In a nutshell, use only semi-colon <strong>or</strong> slash at the end of a SQL (DML or DDL) <strong>and</strong> semi-colon followed by slash at the end of a PL/SQL. See <strong><a href="http://www.dummies.com/how-to/content/the-scoop-on-sql-and-plsql.html" target="_blank">here </a></strong>for the difference between SQL and PL/SQL).<br/><br/>As a personal choice, I normally use "/" for all DDLs (SQL or PL/SQL) and semi-colon for all DMLs. You can have one DDL per file in which case, the "/" is the last character in the file. You can also have multiple DDLs in a file, each separated by "/". Try not to mix DDLs and DMLs in the same file! This may lead to more confusion and errors!<br/><br/><strong>References</strong><br/><br/><a href="http://www.orafaq.com/faq/what_are_the_difference_between_ddl_dml_and_dcl_commands" target="_blank">http://www.orafaq.com/faq/what_are_the_difference_between_ddl_dml_and_dcl_commands</a><br/><br/><a href="http://docs.oracle.com/html/A90108_01/sqcmd.htm">http://docs.oracle.com/html/A90108_01/sqcmd.htm</a><br/><br/><strong>Notes:</strong><br/><br/><hr /><br/><br/><strong>1. </strong><small>See here for e.g., <a href="https://forums.oracle.com/thread/1020117" target="_blank">https://forums.oracle.com/thread/1020117</a></small><br/><br/><strong>2.</strong> <small>When I say "executed", I mean the PL/SQL block that creates the function/procedure is executed - meaning the program is compiled and stored in the database. (Hence the name stored programs.) The actual program execution is done by</small><br/><pre>SELECT <function name> FROM dual;<br/><br/><strong>OR</strong> EXEC <procedure name></pre>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com2tag:blogger.com,1999:blog-785353890590284114.post-48443911415913497052013-06-05T09:20:00.000-07:002013-06-20T22:01:47.213-07:00Source Code Control<h4><strong>Background </strong></h4><br/>I always think, we programmers are like writers and artists. We create software that could be powerful and beautiful. There is no lack of abstraction in our work either!<a href="http://www.toondoo.com/cartoon/6377740"><img class="alignright" style="border:0 none;" title="Click to View Full Size Image" alt="writer" src="http://static.toondoo.com/public/c/a/r/cartoonsv//toons/cool-cartoon-6377740.png" width="288" height="326" border="0" /></a><br/><br/>There is one difference, though. When an Author writes a book, she could revise her work so many times at her desk, but only the final outcome becomes public. Not so with our work! A programmer comes up with several revisions of his/her program before it goes to production. Even so, there may be a bug or something was missed that he/she has to revise it again.<br/><br/>A software could also be "backed out" to a prior version. (Even software giant like <a class="zem_slink" title="Microsoft" href="http://maps.google.com/maps?ll=47.6395972222,-122.12845&spn=0.01,0.01&q=47.6395972222,-122.12845 (Microsoft)&t=h" target="_blank" rel="geolocation">Microsoft</a> does this once in a while. Remember when people rejected <a class="zem_slink" title="Windows Vista" href="http://en.wikipedia.org/wiki/Windows_Vista" target="_blank" rel="wikipedia">Windows Vista</a>, their solution was to go back to XP, the prior version of Windows! I myself went back to <a title="My journey with Ubuntu Contd." href="http://power2build.wordpress.com/2012/08/04/my-journey-with-ubuntu-contd/" target="_blank">Gnome desktop</a> on my <a class="zem_slink" title="List of Ubuntu releases" href="http://en.wikipedia.org/wiki/List_of_Ubuntu_releases" target="_blank" rel="wikipedia">Ubuntu 12.04</a> desktop, because of some issues with <a class="zem_slink" title="Unity (user interface)" href="http://unity.ubuntu.com" target="_blank" rel="homepage">Unity desktop</a>). And of course, someone might point out that something worked in an earlier version, that you will have to dig up the sources for that version, find out what changed and try to re-incorporate that code back into the new version!! I can't imagine an artist having to go through this type of revising of his/her painting! (I agree a book may have editions, but it's always move forward for them.)<br/><br/>So, if you are a software developer you need to worry about keeping track of the history of the program. Not just the history, but all the files that made up a particular version! And to do this we resort to so many ingenious(?) ways. How many times, have you seen files with .bak, .sav, .org etc extensions on a developer's machines? (guilty!). Some may even organize their files in dated directories, so they can go back to a specific date (guilty as well). But when your program is made up of so many files (gone are the days when you wrote all your code in one file!), such rudimentary methods of keeping track of files won't work! Even more so, if you are working (collaborating) with other developers.<br/><h4>Source Code Control or Version Control</h4><br/>Wouldn't it be better if a Software can track these information at the source level, so it's easy to pull up any version of the software sources and rebuild it? Such a software tool exists for over 40+ years and is known variously as Source Code Control Software (<a class="zem_slink" title="Source Code Control System" href="http://en.wikipedia.org/wiki/Source_Code_Control_System" target="_blank" rel="wikipedia">SCCS</a>), Version Control System (VCS), <a class="zem_slink" title="Revision control" href="http://en.wikipedia.org/wiki/Revision_control" target="_blank" rel="wikipedia">Revision Control</a> System (<a class="zem_slink" title="Revision Control System" href="http://www.gnu.org/software/rcs/" target="_blank" rel="homepage">RCS</a>) and even Software Control Management (SCM). Dozens of tools have been developed with these acronyms in their names. Some of the most popular ones over the years are SCCS, RCS, PVCS, Perforce, <a class="zem_slink" title="Microsoft Visual SourceSafe" href="http://www.microsoft.com/" target="_blank" rel="homepage">Visual Source Safe</a> (VSS), Mercury etc. And in the open source arena, we have had CVS, SVN and Git. Please refer to the<span style="text-decoration:underline;"><strong><a title="Comparison of Version Control Software" href="http://en.wikipedia.org/wiki/Comparison_of_revision_control_software" target="_blank"> wiki site</a></strong></span> for comparison of many more tools available on the market. The difference is not only in the names, but in techniques and technology involved. Newer tools support concurrency better.<br/><br/>The least a VCS can do is to keep track of history of files and thus offers reversibility. With various versions saved in the repository, it is only natural that we require the VCS to provide a good compare (diff) utility or at least the capability of using external diff tools. Any type of files can be saved in the repository including binary files. Though, the diff utility is typically geared for text files.<br/><br/>More traditional VCS offers a centralized repository. SCCS, PVCS etc belong to this category. More modern ones offer more distributed approach. With developers from around the world working on open source projects, distributed repositories more sense. And the other characteristics that go with the territory is the concurrency. Again, modern VCS have more and more concurrent support built in. For e.g., CVS refers to Concurrent Version System.<br/><br/><strong>Further Reading</strong><br/><br/>Please refer to the <a title="Eric Sink's site on Source Code Control" href="http://www.ericsink.com/vcbe/index.html">Eric Sink's website</a> (his book is available in PDF format on his site) for detailed discussion on the SCCS tools. This <a title="History of Source Code Control" href="http://www.flourish.org/blog/?p=397" target="_blank">blogger</a> recaps the history of Source Code control in a funny way! And the<a title="Wiki on Source Code Control" href="http://en.wikipedia.org/wiki/Revision_control" target="_blank"> wiki page</a> like always delivers a nice introduction to this topic. Eric Raymond's <span style="text-decoration:underline;"><span style="color:#3366ff;"><a title="Understanding Version Control System" href="http://www.catb.org/esr/writings/version-control/version-control.html#why_vcs" target="_blank"><span style="color:#3366ff;">paper</span></a> </span></span>explains various generations of Version Control Systems nicely.<br/><br/><a href="http://www.ericsink.com/vcbe/index.html" target="_blank">http://www.ericsink.com/vcbe/index.html</a><br/><a href="http://www.catb.org/esr/writings/version-control/version-control.html#why_vcs">http://www.catb.org/esr/writings/version-control/version-control.html#why_vcs</a><br/><a title="History of Version Control" href="https://code.google.com/p/pysync/wiki/VCSHistory" target="_blank">https://code.google.com/p/pysync/wiki/VCSHistory</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-32452202562975966632013-06-05T07:58:00.000-07:002013-06-20T22:01:47.206-07:00Oracle: Forward Slash in SQL*Plus - Take 2<strong>Take 2?</strong><br/>I <a title="Forward Slash in SQL*Plus" href="http://power2build.wordpress.com/2011/12/09/forward-slash-in-sqlplus/" target="_blank"><strong>posted </strong></a>about Forward slash in Oracle SQL before. I even went back and updated it a few times based on the traffic. Believe it or not, I still get a lot of traffic to this blog site solely based on this one little topic!! So, I thought I would spend a little more time on this.<br/><br/>Wordpress has a nice feature in the Stats page that shows the Search engine terms that drove the user to my page. This is what some enters in say, Google and clicks Search. Here are some sample terms on Slash:<br/><br/>sqlplus forward slash<br/>sql forward slash<br/>forward slash in sql<br/>oracle forward slash<br/>oracle sql forward slash<br/>slash in sql<br/>sql slash<br/>.....<br/><span style="color:#ff6600;"><strong>commit vs forward slash</strong></span><br/><strong>oracle pl/sql semi-colon or forward slash</strong><br/><strong>oracle slash before commit</strong><br/><strong>oracle sql commit slash</strong><br/><strong>forward slash in ddl file</strong><br/><br/>and so many variations of these. One thing that caught my attention (in bold) was the confusion user(s) had with commit and Forward slash. They really don't have anything in common, but I guess Oracle behavior in different scenarios could lead one to that. So, I thought of explaining these one more time.<br/><br/><strong>Then Vendor said: Let there be a Database and a (client) Tool</strong><br/><br/>Let's start from the beginning. A database system is typically run on a remote server. To work with it, the vendors - like Oracle, Sybase etc- provide a client tool. Oracle has always come with SQL*Plus as a client tool. This is a simple command line tool, yet powerful. People use it for both interactive and batch use. Oracle also came up with "better" tools eventually, like iSQL*Plus, SQL Developer etc. Similarly, Sybase has Interactive SQL. Apart from these tools, there are plenty of tools available from outside vendors. Some prominent ones for Oracle are Toad, SQLTools, PLSQL Developer etc. The modern tools are typically GUI based and provide plenty of options for interactive usage. Irrespective of that, SQL*Plus is still going strong. Several DBAs prefer this tool to flashy new GUI tools, so whichever tool you use, you may want to make sure your SQL will run SQL*Plus.<br/><br/><strong>Why those Punctuation?<br/></strong>In a GUI tool, you can type up a SQL and click on a button (or press a key), the tool will get the SQL executed (send it to Oracle server, get the results etc). If there are multiple SQLs, you just highlight one and click and go. Now, SQL*Plus is a command line tool. It's text based and often used in both interactive and batch modes. You couldn't really click a button or press a key to execute SQLs. Especially, if it's running SQLs from a file. Then you need a marker to show the end of each SQL, right? That, my friend, is the semi-colon - the statement terminator. Sort of like the period at the end of an English statement. Semi-colon is mentioned as the statement terminator in the SQL Standards books also.<br/><br/>Unfortunately, SQL Standards came much later. Database vendors already came up with their way of telling their tool to execute the SQLs. For e.g., client tools for Sybase and Microsoft SQL Server use <strong>go </strong>command. That makes sense. Type in a SQL. Key in <strong>go</strong> and it goes! (SQL is sent to server to be executed etc). Oracle (SQL*Plus) had a similar command - RUN!! So, if you are running SQLs in a file, you would have a SQL followed by RUN and then the next SQL followed by RUN and so on. This is good. But then Oracle decided to give us the short cuts. Like Forward Slash ("/") for RUN<sup>1</sup>. (Those early days, every byte counted. Why type 3 chars for RUN and why store so many of them in a file? Slash will do it for cheaper price!!).<br/><br/><hr /><br/><br/><strong>1. </strong> <small>Actually RUN = Load & Run and the shortcut for it is "R" while "/" is to just "Run" what's already loaded</small><br/><br/><strong>What does Forward Slash mean?</strong><br/><br/>So, there you have it. Forward Slash executes a SQL before it. If it was left like that, things probably would have been simpler. Oracle also decided to support the statement terminator, semi-colon. (Sybase and other databases did not require this earlier). Now, you have a semi-colon that could get the SQL executed and a Slash that could do the same thing. They are not the same, but served the same purpose of indicating end of a SQL statement in SQL*Plus!! This started the trouble, especially when people started mixing them. Imagine what happens when you have a semi-colon and a forward slash?<br/><br/>Then came PL/SQL. Now, PL/SQL is a programming language in Oracle which can have several embedded SQLs, often separated by you guessed it - semi-colon - the statement terminators. Now you can have a bunch of PL/SQL block in a file, how do you distinguish each block? Don't you need something like a PL/SQL statement terminator? Well, semi-colon is already taken. So, Oracle came up with the ingenious way of reusing a character that already had a purpose - you guessed it, the Forward Slash!!!!! It was meant to be "RUN" the SQL, right? Now, it also has the purpose to terminate PL/SQL. (It could also mean RUN the PL/SQL). That paved the way to all the troubles we face in mixing these up.<br/><br/><strong>Semi-colons and Forward Slashes</strong><br/><br/>That mixture combined with SQL*Plus's own quirks made it worse. In SQL*Plus, you can type a (SQL like) text and end it with semi-colon, it will pass it on to the database. You type a SQL followed by slash ("/"), it will be sent to the server as well. What happens when you have a semi-colon and slash? It gets executed twice. What happens when you have 2 semicolons? You might get an error on the second semi-colon. This is because of the way SQL*Plus works. As soon as you run a SQL, SQL*Plus stores it away in a buffer. Next time you type "/", it re-executes what's already stored in the buffer. Type "/" again, it runs it again. Until you type up a new SQL. You can smell trouble, right? People run into trouble because they don't understand the nuances or simply because they mistyped.<br/><br/>Remember, "/" is a command. So, it has to be on a line by itself. Semi-colon on the other hand, has to be the last character on a line. If there is anything else after that, even a comment (--) or another semi-colon will cause an error!!<br/><br/><strong>DML, DDL etc</strong><br/>When I mentioned SQL above, I meant any type of SQL. SELECT, INSERT, UPDATE, DELETE, CREATE TABLE etc. First 4 deal with data in tables. SELECT is a query. INSERT, UPDATE, DELETE make up the DML. There are whole bunch of SQL statements like CREATE TABLE that actually define the structure of a database object (here TABLE). These are called Database Definition Language (DDL). What we saw above applies to both DMLs and DDLs. You can use semi-colon to end DML statements as well as DDL statements. You can use Forward Slash with DML and DDL statements.<br/><br/>To confuse you a bit, general convention in Oracle world is to use Forward Slash to end DDL statements. This would make sense if you look at the syntax for creating Stored Procedures, Functions etc. These are PL/SQL objects. Remember, I said PL/SQL needs to be ended in Forward Slash? Now that we decided to (have to) use Forward Slash for DDL for Stored programs, why not use it for all DDLs? So, there you have it. This is how we arrived at Semi-colon (";") to end DML statements and Forward Slash ("/") for DDLs.<br/><br/><strong>How did COMMIT enter this discussion about Forward Slash?</strong><br/><br/>Phew! We got through that fine. Now, where does commit come into the picture and why confusion about it? Typically, Oracle has Auto-commit option turned <strong>off</strong> by default. You will have to issue an explicit COMMIT to commit a transaction. (This helps us to have bunch of related DMLs inside a transaction).<br/><br/>Here again there are some short cuts and quirks in the tool that made it more confusing. If you didn't add an explicit COMMIT and simply exit SQL*Plus, the tool will issue the COMMIT for you! And the DDLs do not require explicit COMMIT, as a DDL always issues implicit commits (See <a title="Oracle SQL Gotcha – Commit in DDL" href="http://power2build.wordpress.com/2011/12/09/oracle-sql-gotcha-commit-in-ddl/" target="_blank">here</a>) before and after the actual DDL statement. So, then with DDL it's automatically committed and DDLs typically end with Forward Slash ("/"). Now, I can imagine why the user searched for a connection between Commit and Slash. He/She is probably running a script that has "/" at the end. The user probably ran the script and when SQL*Plus exited, it probably committed. Are you seeing the connection now? Really, there is no connection between "/" and COMMIT. They just look to be related because of quirks and twists in the tool and the specs.<br/><br/><strong>What was all the blabbering about?<br/></strong><br/>In summary, Forward slash is like <strong>go</strong> in SQL Server, but use with caution, as it's not only the RUN command, but could be indirectly construed as a statement terminator as well. Forward Slash ("/") does not commit. SQL*Plus may silently COMMIT your transaction when you exit. But, always add COMMIT and ROLLBACK explicitly.<br/><br/>And Remember, all this is specifically applicable to SQL*Plus. But, since it has become a de facto standard for running SQLs from a file, you make sure to follow these standards to avoid surprises.<br/><br/>And that mumbo-jumbo applies only to Oracle SQL or PL/SQL. Forward slash doesn't have this type of special meaning in other databases!!!!Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com2tag:blogger.com,1999:blog-785353890590284114.post-79436503935778092192013-05-31T07:02:00.000-07:002013-06-20T22:01:47.221-07:00Quick Tip - Oracle 11g Instant ClientOracle 11g has 2 versions of clients, Instant Client which has minimum required files for connecting to Oracle and the full blown (thick?) client which has many drivers and tools like SQL*Plus etc. The full blown version seems to be more for the developer and the instant client seems to be suitable for end users.<br/><br/>Recently, we upgraded to Oracle 11g database. When our desktop admin made changes to a user's PC to connect to our 11g database, one of the applications (in VB) stopped working. Incidentally, he had only installed Oracle instant client for end users, as it uses less space.<br/><br/>The VB application in question actually used OleDB to connect to Oracle. Apparently, the instant client we installed for this user did not include OleDB related files. This is why the connection failed. To make it work, we did the following:<br/><br/>1. Installed Instant Client for Oracle 11g (32bit) (option 1 during installation)<br/><br/>2. After the install we copied the OraOLEDB.dll files into the %ORACLE_HOME%\bin directory and registered it. (using regsvr32).<br/><br/>3. Created the directory C:\Oracle11g\Network\Admin and move over the tnsname.ora. (By default tnsnames.ora is left in Oracle11g folder. If you don't move the file, you need to add/change TNS_ADMIN environment variable to point to the right directory.)<br/><br/>Once we did this, the VB application was able to connect to Oracle 11g through OleDB.<br/><br/><strong>Note:</strong><br/><br/>1. This is the minimum required client installation for Oracle. We didn't install the full blown client, as it comes with SQL*Plus etc which the end user doesn't need.<br/><br/>2. All the data access drivers are bundled in an optional package called ODAC. We didn't want to install all the drivers for the user. So, copied only OleDB driver from here.<br/><br/>3. Apparently, SQL*Plus can be added on to instant client as well. See <a href="http://www.oracle.com/technetwork/database/features/oci/ic-faq-094177.html">here </a>for more information about Oracle instant Client. Question about connection strings has a hint for connection problems similar to ours.<br/><br/><strong>Reference:</strong><br/><a href="http://www.oracle.com/technetwork/database/features/instant-client/index.html">http://www.oracle.com/technetwork/database/features/instant-client/index.html</a><br/><br/><a href="http://www.oracle.com/technetwork/database/features/oci/ic-faq-094177.html">http://www.oracle.com/technetwork/database/features/oci/ic-faq-094177.html</a><br/><br/><a href="http://www.oracle.com/technetwork/topics/dotnet/downloads/index.html">http://www.oracle.com/technetwork/topics/dotnet/downloads/index.html</a><br/><br/><a href="http://support.dundas.com/Dashboard4.Installing_the_Oracle_Data_Provider_for_dot_NET.ashx">http://support.dundas.com/Dashboard4.Installing_the_Oracle_Data_Provider_for_dot_NET.ashx</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-82912765469386964162013-05-31T04:34:00.000-07:002013-06-20T22:01:47.224-07:00Tools: Notepad++ - Syntax HighlightingI've posted about Notepad++ (Npp) tool briefly in a different context. I've been using it for a while and the more I use, the more I discover about the tool. This post is about the syntax highlighting in the tool and how you can customize it.<br/><br/>Npp is simply a text editor. It does the same thing Windows' Notepad does and then some. Hence the name Notepad++. With it, you can open almost any text file. Have you ever tried to open a text file from Unix (or Linux) in Windows' Notepad? All lines will be collapsed into a big jumble? This won't happen in Notepad++. It also allows you to adapt various encoding in file (ASCII/ANSI, UNICODE etc) and thus you won't see box characters (well, most of the time). So, it's a great text editor. But that's not all. What makes it a great tool are it's easy-to-use user interface, it's search capabilities (includes Regex out of the box), plugins etc. If you haven't already tried it, download it from <a href="http://notepad-plus-plus.org/">here</a>.<br/><br/>As developers we tend to expect a lot more from any tool. I myself have used several editors, starting DOS's EDIT, Unix Vi etc. (I still remember <a href="http://en.wikipedia.org/wiki/File:BriefTextEditor.png">Brief Editor </a>of the early 90's made by a company called Underware!) Each had unique capabilities and weaknesses. One thing they were all missing was the Syntax highlighting. For a long time, this was the feature of IDEs/Editors that came with the language. But in the era of extensible editors like Eclipse, we've come to expect Syntax highlighting as a minimum for any text editor.<br/><br/>I have always wondered how the programming tools color their syntax. I can understand a dedicated tool like Powerbuilder. They know their language, so they can just hard code it in their programs. How does a simple editor like Notepad++ do that without having knowledge about each language it supports? These tools provide some kind of mechanism (proprietary or standard) to define various aspects of the syntax and how to highlight them. But, they just don't know anything about the language itself. That is the syntax highlighting in such tools is based on pure syntax and not the semantics of the language. This is the key to understanding some anomalies in highlighting in these tools.<br/><br/>Notepad++ (Npp) is one of the editors based around a standard editing component called <a href="http://www.scintilla.org/ScintillaRelated.html">Scintilla</a>. According to their home page, "<a href="http://www.scintilla.org/ScintillaDoc.html">Scintilla</a> is a free source code editing component. It comes with complete source code and a <a href="http://www.scintilla.org/License.txt">license</a> that permits use in any free project or commercial product.". More on this later. Npp comes with syntax definitions for several languages. But, if your language of choice is not in the list, no worries; you can simply add another "User Defined Language" (UDL). With the latest version (6.2.x), it's improved and called UDL2.<br/><br/>It's the UDL that I am going to talk about in this post. If you open Npp and click on Language menu, you will see a list of languages available. You can then click "Define Your language" option and get going with adding the definition for your language. I used this define the syntax highlighting for Powerbuilder code.<br/><br/><strong>User Defined Language (UDL)</strong><br/><br/><a href="http://power2build.files.wordpress.com/2013/05/npp_udl_dialog.png"><img src="http://power2build.files.wordpress.com/2013/05/npp_udl_dialog.png" alt="" width="546" height="748" class="alignnone size-full wp-image-4866" /></a><br/><br/>Fig 1. Npp UDL dialog<br/><br/>Using this dialog, you can create new syntax definitions, import, export and copy from another language definition. There are several tabs here. We will come to Folder & Default in a minute. Keywords Lists is where you put in the keywords in your language. You can separate the keywords in your language into several groups. For e.g., in my example below, I've 2 lists - one for all the reserved words in Powerbuilder and the second list contains all the types. Separating it thus allows you style your keywords distinctly.<br/><br/><a href="http://power2build.files.wordpress.com/2013/05/npp_udl_powerbuilder.png"><img src="http://power2build.files.wordpress.com/2013/05/npp_udl_powerbuilder.png" alt="" width="550" height="433" class="alignnone size-full wp-image-4868" /></a><br/><br/>Fig 2. Setting up Keyword Lists<br/><br/>When you define an entity - here Keyword list, you can also attach a styler to it. The styler is where you define the fonts and colors for the text for the particular type of entity. For e.g., I've colored PB reserved words in blue as shown in Fig3.<br/><br/><a href="http://power2build.files.wordpress.com/2013/05/npp_udl_pb_kwd1_styler.png"><img src="http://power2build.files.wordpress.com/2013/05/npp_udl_pb_kwd1_styler.png" alt="" width="483" height="258" class="alignnone size-full wp-image-4871" /></a><br/><br/>Fig 3. Styler Dialog<br/><br/>Next Tab in the UDL dialog is the "Comment & Number". This is where you can define the commenting symbols in your language. For comments they have 2 styles. Line Style which is a single comments. In PB (C, C++, Java etc) single line comments are identified // at the beginning of the text. Such comments can be at the beginning or end of the line or by themselves.<br/><br/>The second style is C style languages is the multi-line comments, anything in between /* and */. Here is a screenshot of the comment section. As you can see, there is a lot more you can do with it. In the styler for the comments, you can also specify about nested comments.<br/><br/><a href="http://power2build.files.wordpress.com/2013/05/npp_udl_pb_comment.png"><img src="http://power2build.files.wordpress.com/2013/05/npp_udl_pb_comment.png" alt="" width="509" height="327" class="alignnone size-full wp-image-4867" /></a><br/><br/>Fig 4. Setting up Syntax highlighting for comments<br/><br/>Similarly, you can highlight numbers.<br/><br/>The last tab is the <strong>Operators & Delimiters</strong>. This is where you list out the operators and any delimiters in the language you are defining. Operators are the symbols like +, -, /, * etc. Comma (,), semi-colon(;), colon(:), pipe (|) are examples of delimiters. Again, use the stylers to format them.<br/><br/>I will now get back the first tab - <strong>Folders & Default</strong>. Default is simple - This the normal text that is not defined in any of the tabs above - i.e., text that is not a keyword, comment, operator or a number. To define this, just click on the Default Style-> Styler button.<br/><br/>The folders section is a bit more interesting. Have you ever seen <a href="http://en.wikipedia.org/wiki/Code_folding">code folding</a>? Eclipse has it. I believe, Visual Studio does. Powerbuilder itself does not offer it. It's the feature in some editors that let you selectively hide sections of code, so you can see the "big picture". For e.g., if you are working with a multi-level nested IF statements, you can hide inner ones to see what the outer most if/else does. You typically see code folding available for any control structures (let's call it block definers) in the programming language - like IF, CASE, FOR, DO, {, } etc. These typically have an opening and closing marker text or symbol. The code folding also helps in such cases, to see if any block is missing closures.<br/><br/>In this tab, Npp UDL offers 2 styles - one that doesn't require separators and the other that does. This is a bit confusing and I had to try both to understand. The one that doesn't require separator are symbols like {, }, (, ) etc. Even if you have text touching them, they are identifiable. You put them in the first style. The second style, that needs separators (typically space), will contain all the other block definers. For e.g., in PB we have IF...ELSE...END IF to define a IF block. The opener is the IF, END IF is the closer. ELSE happens to be the middle (Earlier versions of UDL only supported open and close. So, upgrade to the latest version of Npp, if you haven't already done so). Notice END IF is actually 2 words; in such cases surround them in Double quotes.<br/><br/>Folding in comments allows you to put specific comments in your code, that you can use for folding the section. We often comment out whole block of codes to test portions of code. Just add marker texts (open, middle, close) defined earlier to such comments, you will be able to just hide the whole block.<br/><br/><a href="http://power2build.files.wordpress.com/2013/05/npp_udl_pb_code_folding.png"><img src="http://power2build.files.wordpress.com/2013/05/npp_udl_pb_code_folding.png" alt="" width="545" height="599" class="alignnone size-full wp-image-4870" /></a><br/><br/>Fig 5. Code Folding in Npp<br/><br/>Of course, you can add styles in the stylers.<br/><br/>Here is a sample PB code in Npp, after I created the UDL "Powerbuilder" and applied to the file opened.<br/><br/><a href="http://power2build.files.wordpress.com/2013/05/npp_edit_pb_code.png"><img src="http://power2build.files.wordpress.com/2013/05/npp_edit_pb_code.png" alt="" width="1050" height="835" class="alignnone size-full wp-image-4869" /></a><br/><br/>Fig 6. Editing PB code in Npp<br/><br/><strong>Note:</strong> When I added code folding the syntax highlighting for the keywords in Code folding (IF, END IF, CHOOSE CASE etc) seem to disappear. It may be a bug in the current version of Npp. I will check and post back here.<br/><br/>[gallery include="4872,4873,4874,4875,4876,4877"]Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-61313945109892785652013-04-15T11:43:00.000-07:002013-06-20T22:01:47.216-07:00Top N subcategories<a href="http://searchoracle.techtarget.com/answer/Top-N-subcategories#.UWyssyXhDxY.wordpress">Top N subcategories</a>.Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-78221014616728633362013-04-15T11:42:00.000-07:002013-06-20T22:01:47.218-07:00Oracle SQL - UNION ALL<p><a href="http://searchoracle.techtarget.com/answer/Using-SQL-SELECT-and-SQL-UNION-ALL-statements-to-sort-numbers?utm_medium=EM&asrc=EM_ERU_21322748&utm_campaign=20130415_ERU%20Transmission%20for%2004/15/2013%20%28UserUniverse:%20627644%29_myka-reports@techtarget.com&utm_source=ERU&src=5121532" title="Oracle SQL - UNION ALL">Oracle SQL - UNION ALL</a></p><p>Here is another TechTarget article about Oracle SQL. This time about using UNION ALL to combine results of 2 SQLs to sort numbers.</p>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-82563046280374295022013-04-11T10:53:00.000-07:002013-06-20T22:01:47.222-07:00Windows Automation Tools/ScriptsAfter the successful run with JMeter earlier to load test our web pages, I have been on a mission to find an open source tool to automate our GUI application. So far, I have not found any good (free) open source tool. But, I stumbled upon several nice tools and scripting languages to automate windows GUI application.<br/><br/>JMeter was very good for the Web. Since each web page can be generated (HTML), the tool could simulate page to page navigation well. If only we can do this for the GUI screens? It's all compiled code and a GUI doesn't look like it can be tampered with (until I found below scripts). So, you essentially need to work on it like a black box, so you can automate a GUI application only by capturing it's external interfaces - mouse clicks, keys types and windows messages to and from the application.<br/><br/><strong>Sikuli</strong><br/><br/>I started with <a href="http://sikuli.org/"><strong>Sikuli</strong></a>. This is a great tool/script. It's Python based and lets you capture screen shots of different parts of the screen and drive your script by looking for these images. They call it Computer Vision. You can build a library of standard images (For e.g., Button with the label OK) and reuse them across the application(s). As long as your application has GUI standards, this will work out great. It's a great concept and it seems to work decently.<br/><br/>[caption id="attachment_4847" align="alignleft" width="1152"]<a href="http://power2build.files.wordpress.com/2013/04/sikuli_ide.png"><img class="size-full wp-image-4847" alt="sikuli ide" src="http://power2build.files.wordpress.com/2013/04/sikuli_ide.png" width="1152" height="864" /></a> Fig 1. Sikuli IDE. Notice the version # in the title r930 is the latest and stable version.[/caption]<br/><br/>Sikuli IDE lets you capture screenshots from within and use these in functions For e.g., wait function, waits for a screen matching the screenshot to appear. After you create your script in IDE and save it, the underlying script is actually saved as a Python script, with the all the images saved.<br/><br/>[caption id="attachment_4848" align="alignnone" width="588"]<a href="http://power2build.files.wordpress.com/2013/04/siklui_ltx_script.png" target="_blank"><img class="wp-image-4848 " alt="siklui ltx script" src="http://power2build.files.wordpress.com/2013/04/siklui_ltx_script.png" width="588" height="493" /></a> Fig 2. Corresponding Python Script[/caption]<br/><br/>Sikuli has a full array of built in (global) functions. See <a href="http://doc.sikuli.org/globals.html">documentation </a>for more.<br/><br/><strong>Note: </strong>Above script has UDF - user defined functions.<strong> </strong>When you record script through AutoIt3 Recorder, it will have one long script. I've divided these into functions.<br/><br/>Sikuli is a great tool. It deserves more research and I reserve it for future. I am more of a scripting guy and I like text based scripts. (It's easier to find and replace globally and you can even write a script to generate another). So, I continued my search for my text based scripting language. Also, I had some issue with the installation. Though it worked initially, the runtime (a batch file running Java jar) kept getting corrupted for some reason. Then I tried their portable version, it seemed to work OK so far. But, if you are having issues with Sikuli installation, please see <a href="https://answers.launchpad.net/sikuli/+faq/2005">here,</a> <a href="https://answers.launchpad.net/sikuli/+question/99224">here</a>, <a href="https://answers.launchpad.net/sikuli/+question/139787">here,</a> here. I did not see any recorder for Sikuli, though I saw references to one all over the web.<br/><br/><strong>AutoIt3</strong><br/>Then I landed on <strong><a href="http://www.autoitscript.com">AutoIt</a></strong>. It's great tool. I've really grown to like it. It has a BASIC kind of language (actually more like PERL, C etc) and lets you program everything from mouse clicks to taking complete control of a GUI application window. Apparently original AutoIt reached version 2 and stopped. There is a new branch called AutoIt3. This version is bigger and better than previous versions. See here for full <strong><a href="http://www.autoitscript.com/autoit3/docs/intro/dev_history.htm">history</a></strong> of the product.<br/><br/>AutoIt3 scripts can be edited in integrated <a href="http://www.scintilla.org/SciTERegEx.html">Scite Editor</a>. All popular editors like Notepad++ also support Autoscript syntax. AutoIt3 comes with a Recorder as well. I was able to run our application and record it into a script that I could use repeatedly. I am just beginning to explore this tool. I will post more as I explore the tool further.<br/><br/><img class="wp-image-4849 alignnone" style="font-family:Helvetica, Arial, 'Lucida Grande', sans-serif;font-size:16px;" title="AutoIt3 Script in NPP" alt="au3 scripts in notepadpp" src="http://power2build.files.wordpress.com/2013/04/au3_scripts_in_notepadpp.png" width="780" height="557" />Fig 3 AutoIt3 Script edited in Notepadd++<br/><br/>When you install AutoIt3, the commands are integrated into Windows Explorer menu. You can edit or Run script from here. Edit opens Scite Editor by default.<br/><br/>[caption id="attachment_4850" align="aligncenter" width="300"]<a href="http://power2build.files.wordpress.com/2013/04/au3_explorer_menu.png"><img class=" wp-image-4850 " alt="au3 explorer menu" src="http://power2build.files.wordpress.com/2013/04/au3_explorer_menu.png?w=300" width="300" height="111" /></a> Fig 4. Au3 Commands integrated into Windows Explorer menu[/caption]<br/><br/><strong>Autoit3 Samples</strong><br/><br/>AutoIt site has a very good forum with lots of skilled people supporting it. A section called "<a href="http://www.autoitscript.com/forum/forum/9-example-scripts/"><strong>Example Scripts</strong></a>" provides lots of good sample scripts. For a real dive into AutoIt3, take a look at a script called <a href="http://www.autoitscript.com/forum/topic/126716-control-viewer-autoit-window-info-tool/">Control Viewer</a>. The author has taken the pains to develop it in AutoIt3 itself. It shows you how to create GUI in Autoscript, and using this you can identify controls (widgets) on any windows application currently running. (AutoIt3 comes with AutoInfo which has similar functionality, but CV has it better!). For some reason the author has discontinued it, but you can down the EXE and the scripts from the main page.<br/><br/>[caption id="attachment_4851" align="aligncenter" width="209"]<a href="http://power2build.files.wordpress.com/2013/04/au3_cv_script_gui.png"><img class="wp-image-4851 " alt="au3 cv script gui" src="http://power2build.files.wordpress.com/2013/04/au3_cv_script_gui.png?w=209" width="209" height="300" /></a> Fig 5. Control Viewer window - GUI built in AutoIt3 script![/caption]<br/><br/><strong>AutoHotKey (AHK)</strong><br/>The third product I looked at, but haven't done much with, is <a href="http://www.autohotkey.com/"><strong>AutoHotKey (AHK)</strong></a>. Apparently it combines strengths of 2 different programs, AutoIt (older version) and Hotkey. One immediate observation about AHK scripts was that it didn't look as structured as AutoIt scripts. But, I could be wrong. There are debates about AutoIt vs AHK. See here for a comparison, though somewhat old. AutoIt3 wins hands down for me at the moment.<br/><br/>I will post more as I explore any of these tools for use in test automation. If you are interested, here are some interesting links about the topic:<br/><br/><strong>References</strong><br/><br/><a href="http://www.autoitscript.com/site/">http://www.autoitscript.com/site/</a><br/><br/><a href="http://www.autoitscript.com/autoit3/docs/">http://www.autoitscript.com/autoit3/docs/</a><br/><a href="http://www.autoitscript.com/autoit3/docs/intro/dev_history.htm">http://www.autoitscript.com/autoit3/docs/intro/dev_history.htm</a><br/><a href="http://www.autoitscript.com/forum/forum/9-example-scripts/">http://www.autoitscript.com/forum/forum/9-example-scripts/</a><br/><br/><a href="http://www.sikuli.org/">http://www.sikuli.org/</a><br/><a href="http://www.javaedge.com/pdf/Sikuli-Gennadi-Zimmerman.pdf">http://www.javaedge.com/pdf/Sikuli-Gennadi-Zimmerman.pdf</a><br/><a href="http://tux2323.blogspot.com/2011/06/sikuli-gui-test-automation-with-java.html">http://tux2323.blogspot.com/2011/06/sikuli-gui-test-automation-with-java.html</a><br/><a href="http://groups.csail.mit.edu/uid/other-pubs/uist2011-sikuli-guide.pdf">http://groups.csail.mit.edu/uid/other-pubs/uist2011-sikuli-guide.pdf</a><br/><a href="http://code.google.com/p/simplesikuli/">http://code.google.com/p/simplesikuli/</a><br/><a href="http://stackoverflow.com/questions/2125839/has-anybody-used-sikuli-for-testing-their-gui-based-apps">http://stackoverflow.com/questions/2125839/has-anybody-used-sikuli-for-testing-their-gui-based-apps</a><br/><br/><a href="http://www.autohotkey.com/">http://www.autohotkey.com/</a><br/><a href="http://paperlined.org/apps/autohotkey/autoit_and_autohotkey.html">http://paperlined.org/apps/autohotkey/autoit_and_autohotkey.html</a><br/><br/><a href="http://www.scintilla.org/SciTERegEx.html">http://www.scintilla.org/SciTERegEx.html</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-23804056188776884142013-03-25T01:19:00.000-07:002013-06-20T22:01:47.202-07:00Link: Confusion around PL/SQL to_date and to_char functionsSee the <a title="TechTarget" href="http://www.techtarget.com/" target="_blank">TechTarget</a> post below:<br/><br/><a href="http://searchoracle.techtarget.com/answer/Confusion-around-PL-SQL-to_date-and-to_char-functions#.UVBqwqxLObc.wordpress">Confusion around PL/SQL to_date and to_char functions</a>.Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-72409574314916412662013-02-05T14:04:00.000-08:002013-06-20T22:01:47.217-07:00Quick Tip: tnsping<strong>Tnsping</strong> is a nice little utility that comes with Oracle. Lot of developers don't know about this. If you have SQL*Plus available on your machine, chances are you also have tnsping. You can use this tool to troubleshoot Oracle connectivity issues, sort of like ping for TCP/IP. To use it, just type<br/><pre>tnsping <Service Alias> [count]</pre><br/>Where Service Alias is defined in <a href="http://www.orafaq.com/wiki/Tnsnames.ora" target="_blank">tnsnames.ora</a>.<br/><br/>Typically, if tnsping returns an error, chances are the alias doesn't exist in tnsnames.ora file or a typo. Just add/correct that in the file, you will be able to ping the Oracle instance.<br/><br/>On a Windows system, this is where an introduction to tnsnames.ora would end. But not on Unix.<br/><br/><strong>Gotcha on Unix</strong><br/><br/>When we had connectivity issue with Oracle 11g database this morning, we kept getting errors while trying to tnsping.I logged in with the oracle id and checked the contents of tnsping.ora file. It looked OK. Also, with this id, I was able to login to Oracle without any problem. This was a puzzle.<br/><br/>Then I remembered the file level permission on *nix systems. I looked at the permissions attribute of the file, tnsnames.ora (If you do a ls -l on Unix, it shows you that). Bingo! The file had didn't have "read" access for Other (public).<br/><pre>-rw-rw-<strong>r</strong>-- tnsping.ora</pre><br/>In this case, even though the file was there, it wasn't "visible" to other id's, because of missing read permission. When I added that (r in bold), the other user was able to tnsping and connect to Oracle finally.<br/><br/><strong>Note on connectivity</strong><br/><br/>If tnsping is successful, it merely tells us that the SQL*Net listener is running correctly on the server side. This doesn't guarantee that the database itself is running. You need to login to find that out.<br/><br/><strong>References</strong><br/><ol><br/> <li><a href="http://edstevensdba.wordpress.com/2011/02/27/tnsping-101/" target="_blank">http://edstevensdba.wordpress.com/2011/02/27/tnsping-101/</a></li><br/> <li><a href="http://www.orafaq.com/wiki/Tnsnames.ora" target="_blank">http://www.orafaq.com/wiki/Tnsnames.ora</a></li><br/> <li><a href="http://www.toadworld.com/KNOWLEDGE/KnowledgeXpertforOracle/tabid/648/TopicID/TDT2/Default.aspx" target="_blank">http://www.toadworld.com/KNOWLEDGE/KnowledgeXpertforOracle/tabid/648/TopicID/TDT2/Default.aspx</a></li><br/> <li><a href="http://docs.oracle.com/cd/B19306_01/network.102/b14212/connect.htm#sthref1535" target="_blank">http://docs.oracle.com/cd/B19306_01/network.102/b14212/connect.htm#sthref1535</a></li><br/> <li><a href="http://www.dartmouth.edu/~rc/help/faq/permissions.html" target="_blank">http://www.dartmouth.edu/~rc/help/faq/permissions.html</a></li><br/></ol>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-45643473126597671532013-02-05T11:20:00.000-08:002013-06-20T22:01:47.210-07:00Gotcha: Oracle11g Password Case SensitivityWe are in the process of upgrading to <a class="zem_slink" title="Oracle Database" href="http://www.oracle.com/" target="_blank" rel="homepage">Oracle 11g</a>. Today, a friend at work asked about an issue he had with connecting to Oracle 11g from Unix. He was able to connect fine in SQL*Tools, SQL*Plus etc, but not when connecting from a script. The script kept failing with invalid user name or password error. The difference was that the script actually used an encrypted password and decrypted it using <a title="CCrypt" href="http://ccrypt.sourceforge.net/" target="_blank">ccrypt</a> utility.<br/><br/>We tried the decrypt option on command line, and it turns out the password was all upper case. Same script used to work in 10g. Next, we tried SQL*Plus with upper case password. It failed while the same thing worked in Oracle 10g. There we realized that Oracle 11g passwords may be <a class="zem_slink" title="Case sensitivity" href="http://en.wikipedia.org/wiki/Case_sensitivity" target="_blank" rel="wikipedia">case sensitive</a>.<br/><br/>A beautiful <a href="http://www.oracle-base.com/articles/11g/case-sensitive-passwords-11gr1.php" target="_blank">post</a> about the same, confirmed this. Apparently, passwords are case sensitive in Oracle 11g by default. This can be overridden as mentioned in the above link. Just to be complete, I am including that here:<br/><pre>ALTER SYSTEM SET SEC_CASE_SENSITIVE_LOGON = FALSE;</pre><br/>You can also look at <a href="http://www.oracle-base.com/articles/11g/case-sensitive-passwords-11gr1.php">Oracle page</a> about authentication for more details. <a href="http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:708040800346734217" target="_blank">AskTom</a> talks about this change.<br/><br/>It was a real gotcha for us today.<br/><br/><strong>References</strong><br/><a href="http://www.oracle-base.com/articles/11g/case-sensitive-passwords-11gr1.php" target="_blank">http://www.oracle-base.com/articles/11g/case-sensitive-passwords-11gr1.php</a><br/><a href="http://docs.oracle.com/cd/B28359_01/server.111/b28320/initparams211.htm" target="_blank">http://docs.oracle.com/cd/B28359_01/server.111/b28320/initparams211.htm</a><br/><a href="http://docs.oracle.com/cd/B28359_01/network.111/b28531/authentication.htm#DBSEG3225" target="_blank">http://docs.oracle.com/cd/B28359_01/network.111/b28531/authentication.htm#DBSEG3225</a><br/><a href="http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:708040800346734217" target="_blank">http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:708040800346734217</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-48404353809598399662013-02-02T15:13:00.000-08:002013-06-20T22:01:47.212-07:00Ubuntu in action... - NDAS<img class="alignleft" alt="" src="http://www.iocellnetworks.com/neo/images/netdisksolo/netdisksoloimage01.png" width="325" height="325" />Last week, my Windows PC stopped working. (I have to look into it later). At the moment, I am working with my taxes and I needed some files from the old machine, desperately. Luckily, I've been backing up. For this, I use <a title="IOCell NetDisk" href="http://www.iocellnetworks.com/neo/index.php/2011-07-07-08-11-37/netdisk-solo" target="_blank">IOCell Networks (also Ximeta)<strong><sup>1</sup></strong>, NetDisk</a>. NetDisk is a <a href="http://en.wikipedia.org/wiki/Network_Direct_Attached_Storage" target="_blank">Network Direct Attached Storage</a> (NDAS) device. This seems to be a combination of <a href="http://en.wikipedia.org/wiki/Network-attached_storage" target="_blank">Network Attached Storage</a> (NAS<a href="http://en.wikipedia.org/wiki/Network-attached_storage" target="_blank">)</a> and <a class="zem_slink" title="Direct-attached storage" href="http://en.wikipedia.org/wiki/Direct-attached_storage" target="_blank" rel="wikipedia">Direct Attached Storage</a> (DAS). As a device, NetDisk is really an external <a class="zem_slink" title="Disk enclosure" href="http://en.wikipedia.org/wiki/Disk_enclosure" target="_blank" rel="wikipedia">hard disk enclosure</a> that is network enabled and you add hard disk (I added a 1TB drive to it). The difference from other external drive enclosures is that this is network enabled, so once plugged into Ethernet, it can be accessed from any PC on the network. And Direct Access because, you can just plug in the device directly to Ethernet, not through any computer or server. To add security, they have added a software layer (IOCell/Ximeta NDAS) that you need to install on each PC to access the drive. You will have to register the software with the specific serial # of the NetDisk. Thus, only specific PCs with the right software and serial # register can access the drive. I had installed this software on my Vista machine and thus worked like a charm. (This post relates only to the Linux version of this software. If you need help with Windows version, please refer to their <a href="www.ximeta.com/files/nd_winmanual_us.pdf" target="_blank">user manual</a><a class="pplsrsla" id="srsl_1"></a><span class="pplsrsl">).</span><br/><br/>Now, at this critical moment, when I needed to access the back up, I couldn't. My main PC is dead. My wife's laptop runs Windows 7 and is not capable of running Ximeta NDAS software (It freezes - even the <a href="http://www.ximeta.com/web/webTemp/html/zbxe/?document_srl=11145" target="_blank">version 2.72 </a>they suggest to fix the issue). Then my other PC, where I am typing this from, is running Ubuntu 12.04 for which Ximeta does not have a driver or programs for NDisk. At least, not any installation binaries!<br/><br/>But there was hope. Luckily, Ximeta provides (open source) source code at <a href="https://github.com/iocellnetworks/ndas4linux" target="_blank">https://github.com/iocellnetworks/ndas4linux/tree/master/ <span style="color:#000000;">for Linux</span>. D</a>ownloaded the source code (If you go to zip tab in the above page, it will download a file named ndas4linux-master.zip. The source directory has several versions. Which version to use depends on your kernel version. (To find your kernel's version, you can use <strong>uname -r</strong>). For Ubuntu 12.04, I used version 3.2.0. Though I work with large systems at work, I never venture to compile big programs at home. This time, it's different. Desperate situation require desperate measures. I ventured on compiling the driver for <a class="zem_slink" title="Linux" href="http://en.wikipedia.org/wiki/Linux" target="_blank" rel="wikipedia">Linux</a> myself. I was amazed. Huge set of files got compiled while I watched without major issues. Once the source code is compiled, we need to install and load it. Here is a list of things to do from their <a title="Ximeta docs" href="https://github.com/iocellnetworks/ndas4linux/tree/master/3.2.0/doc" target="_blank">documents</a>:<br/><ol><br/> <li>Download a source tarball.</li><br/> <li>Unpack</li><br/> <li>CD into the right folder and run make</li><br/> <li>Change to root or sudo <strong>make install</strong></li><br/> <li>Start ndas as root</li><br/> <li>As root, or by sudo, register and enable ndas devices with ndasadmin</li><br/></ol><br/><div id="LC33"> Apart from the documents provided, each directory inside the main folder has a README. Read these for instructions on how and where to compile. <a title="Build NDAS Software for Linux" href="https://github.com/iocellnetworks/ndas4linux/wiki/How-to-build,-install-and-remove-.deb-ndas-packages" target="_blank">Here</a> is a little better instructions on how to build it.</div><br/><div></div><br/><div><strong>Building the software</strong></div><br/><div></div><br/><div>This is really confusing, as there are many directories and many make files. To do this correctly, you need to be in the right directory. Change to <strong>ndas4linux-master/3.2.0/doc</strong> (I use 3.2.0; change to the right version) and read</div><br/><div>how_to_export.txt first. Here is the summary of how to compile:</div><br/><div></div><br/><pre>cd ndas4linux-master/<strong>3.2.0<br/></strong>make linux-rel</pre><br/><div>This creates a new directory, ndas4linux-master/3.2.0/<strong>build_x86_linux</strong>. linux-rel is one of the options. This is the final version. But, you can also build dev or debug versions. Take a look at the doc files.</div><br/><div></div><br/><div><strong>Installing the software</strong></div><br/><div>Once you compiled, you need to install the driver and start it. Below commands do that.</div><br/><pre>cd ndas4linux-master/3.2.0/build_x86_linux/ndas-3.2.0.x86<br/>make</pre><br/><div>The make command compiles so many files and actually builds <strong>ndasadmin</strong> and "installs" in /usr/sbin. This is the command we will use to mount NetDisk.</div><br/><div></div><br/><div>ndasadmin has several options. Please take a look <a title="ndasadmin" href="https://github.com/iocellnetworks/ndas4linux/wiki/How-to-use-ndasadmin#wiki-start" target="_blank">here</a>. To run this command, you need to be root or use sudo. If you just type sudo <strong>ndasadmin</strong> on command prompt, it will display various options available.</div><br/><div></div><br/><strong>Starting ndas <a class="zem_slink" title="Device driver" href="http://en.wikipedia.org/wiki/Device_driver" target="_blank" rel="wikipedia">Driver</a></strong><br/><br/>After building and installing the driver software, you need to start the driver before you can use the device.<br/><pre>sudo ndasadmin start</pre><br/><div><strong> </strong></div><br/><div><strong>Mounting the <a class="zem_slink" title="Device file" href="http://en.wikipedia.org/wiki/Device_file" target="_blank" rel="wikipedia">Device</a></strong></div><br/>You also need to register the device serial #, before you can mount the device volumes. The readme file, how_to_use_ndasadmin.txt inside the version folder (3.2.0 in my case) lists the steps to install and run NDAS software. To register the device, you need to find the serial number (and write key) of the NetDisk device. These are available at the bottom of the box.<br/><div></div><br/><pre>sudo ndasadmin register <SERIAL_NUMBER> -n <strong>NetDisk1</strong><br/>sudo ndasadmin enable -s 1 -o r</pre><br/><div>brings up the NetDisk volumes. NetDisk1 is just name I gave to the device. This can be anything. Also, register option have a slight variation with Serial # and Write key. See the readme file mentioned above.</div><br/><div></div><br/><div><br/><br/>First, I didn't see them come up. Then I found them listed in "Places" option in Gnome desktop menu. To see the device listing, you can use the following command:<br/><div></div><br/><pre>cat /proc/ndas/devs</pre><br/></div><br/><div></div><br/><div><br/><br/>To disable/unmount the device, use the following commands:<br/><div></div><br/><pre>sudo ndasadmin unregister -n NetDisk1<br/>sudo ndasadmin disable -s 1 -o r</pre><br/></div><br/><div><strong>Further Help</strong></div><br/><div>Like I said earlier, the doc folder inside the version you are working with (<strong>ndas4linux-master/3.2.0/doc</strong>) actually contains all the instructions. Here are some of the files in that directory.</div><br/><div style="padding-left:30px;">how_to_build.txt</div><br/><div style="padding-left:30px;">how_to_export.txt</div><br/><div style="padding-left:30px;">how_to_use_ndasadmin.txt</div><br/><div></div><br/><div>Note that the software seems to be constantly evolving, so it may not support all the functionalities, the actual device can support. And if you need more help, here are some links if you want to learn more about Iocell Ximeta on Linux.</div><br/><div></div><br/><ol><br/> <li><a href="http://ndas4linux.iocellnetworks.com/" target="_blank">http://ndas4linux.iocellnetworks.com/</a></li><br/> <li><a href="https://github.com/iocellnetworks/ndas4linux/wiki/How-to-export-NDAS-source-for-different-architectures" target="_blank">https://github.com/iocellnetworks/ndas4linux/wiki/How-to-export-NDAS-source-for-different-architectures</a></li><br/> <li><a href="http://ndas4linux.iocellnetworks.com/trac/index.cgi/browser/ndas4linux/release" target="_blank">http://ndas4linux.iocellnetworks.com/trac/index.cgi/browser/ndas4linux/release</a></li><br/> <li><a href="http://ndas4linux.iocellnetworks.com/kermit/index.cgi/wiki/Usage" target="_blank">http://ndas4linux.iocellnetworks.com/kermit/index.cgi/wiki/Usage</a></li><br/> <li><a href="http://ndas4linux.iocellnetworks.com/kermit/index.cgi/wiki/Usage" target="_blank">http://ndas4linux.iocellnetworks.com/kermit/index.cgi/wiki/HowToBuildDEB</a></li><br/></ol><br/><br/><hr /><br/><br/><sup><strong>1 IOCell acquired Ximeta's NDAS; so you will see the names used interchangeably here and the web. See <a title="IOCell acquires Ximeta" href="http://www.storagenewsletter.com/news/mergeracquisition/iocell-networks-ximeta-ndas" target="_blank">here</a></strong></sup>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-78810703557169368212013-02-02T06:06:00.000-08:002013-06-20T22:01:47.220-07:00SQLite DatabaseI've been working with different types of Databases over the years from dBase to B-Trieve to Oracle enterprise database. Along the way, I looked at or picked up quite a few - Integra SQL, Informix, MS Access, MySQL, SQL Server(s), Watcom SQL/Sybase SQL Anywhere, DB2, DB2/400 etc. While SQL databases are good, I miss the simplicity of dBase or B-Trieve files. After many years now, I think I found the lost love now!! SQLite!<br/><br/>In my intro I talk about "beautiful software". This is how I felt about this software. Installation is simple, you just download and unzip 1 EXE - sqlite3.exe. (or install in Linux using apt-get or yum). Done! When you are ready to create a database, you type sqlite3 <database name>. That's it!!! You want to work with the same database again, just type,<br/><pre>sqlite3 <database name></pre><br/>The file size is what amazes me - it's a mere 50k. They've really packed a lot inside it. For one, it supports almost full (basic) SQL. I was able to actually run SQLs meant for Oracle in Sqlite. The command line utility achieves what other big databases didn't. SQL*Plus can learn a lot from this tool. And it's highly portable - in fact, I created a database in windows at work and brought home to Linux. It works like a charm.<br/><br/>For one, they differentiated the tool commands by .prefixing with period (.). So, you want help, just type .help. You want to see the tables in the database you just opened, just type .tables.<br/><br/>What impressed me with the tool, other than the simplicity is, that it provides table export imports naturally. Commands?<br/><pre>.import <filename> <table></pre><br/>What about exporting table or query output? Just use,<br/><pre>.dump [<tablename>]<br/><br/>.output <filename></pre><br/>Don't you want to export to CSV or HTML etc.? Sure, just use <strong>.mode.</strong> Then, you want a different separator? Use .separator. (Though HTML mode only dumps the tags for table rows, as it's output is often embedded in CGI outputs, but you can always add the missing tags, right?). Have you tried exporting a table from Oracle SQL*Plus? It's like extracting teeth, so we end up writing programs/procedures to do it.<br/><br/>True, Oracle (and other databases) is much larger and has a lot more bells and whistles, but for a simple (personal) database task, SQLite is the natural choice. Granted, this is just file based database, but smaller tasks this is a great tool to have in your toolbox. My specific task this time, is to import the log files into a database table, so I can query and procedure usage reports and charts for our recently re-factored web app. In the past, I used MS Access for this type of activity, but with a slow machine I have, Access doesn't fair well with hundreds of thousands of log records.<br/><br/>I've only scratched the surface with SQLite. It seems to be used in embedded applications. There is an API available to work with it C/CPP. Python (Perl, PHP etc) come with native support for SQLite. It's a pleasure to work with SQLite in Python. More on this later. Apart from these, SQLite offers extensibility by allowing to load (.load) external modules. You can also use Virtual Tables (like Oracle External table) to access a CSV file directly.<br/><br/>I can go on and on, but I will stop ranting and raving about the greatness of this software. If this post made you want to look up more on SQLite, please try the links below.<br/><ul><br/> <li>http://www.sqlite.org</li><br/> <li>http://en.wikipedia.org/wiki/SQLite</li><br/> <li>http://www.sqlite.org/whentouse.html</li><br/> <li>http://www.sqlite.org/selfcontained.html</li><br/> <li>http://www.askyb.com/sqlite/learn-sqlite-in-1-hour/</li><br/> <li>http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles</li><br/> <li>http://code.btbytes.com/2010/09/16/sqlite-csv-loading-using-virtual-table-extension.html</li><br/> <li>http://docs.python.org/2/library/sqlite3.html</li><br/> <li>http://www.sqlite.org/vtab.html</li><br/> <li>http://www.drdobbs.com/article/print?articleId=202802959&siteSectionName=database</li><br/> <li>http://www.yolinux.com/TUTORIALS/SQLite.html</li><br/> <li>http://robert.accettura.com/blog/2008/02/26/and-the-winner-for-most-embedded-db-is-sqlite/</li><br/></ul><br/> Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-74257071267212509352013-01-22T15:06:00.000-08:002013-06-20T22:02:31.607-07:00Stumbled Upon: New Human Species DiscoveredThis is from 2010, but interesting read<br/><br/><a title="New Human Species Discovered:" href="http://www.sciencedaily.com/releases/2010/03/100325100848.htm" target="_blank">http://www.sciencedaily.com/releases/2010/03/100325100848.htm</a>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-42606344574087538692013-01-22T15:02:00.000-08:002013-06-20T22:02:31.596-07:002012 in review - Auto generated Wordpress ReportThe WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.<br/><br/><a href="http://power2build.wordpress.com/2012/annual-report/"><img alt="" src="http://www.wordpress.com/wp-content/mu-plugins/annual-reports/img/2012-emailteaser.png" width="100%" /></a><br/><br/>Here's an excerpt:<br/><blockquote>600 people reached the top of Mt. Everest in 2012. This blog got about <strong>4,900</strong> views in 2012. If every person who reached the top of Mt. Everest viewed this blog, it would have taken 8 years to get that many views.</blockquote><br/><a href="http://power2build.wordpress.com/2012/annual-report/">Click here to see the complete report.</a><br/><br/><strong>SamV:</strong> Wasn't exactly a good year, but this is the year, this blog is turning around. I am finally seeing some visitors. I am still in the process of improving this. If you happened to stop by here and have some ideas as to how to improve the user experience, please feel free add as comments. I appreciate it. Have a wonderful 2013!!!Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-83227257251175046662013-01-22T14:56:00.000-08:002013-06-20T22:02:31.612-07:00Quick Tip: Java Stored Procedures in Oracle - Reading ContentThis post is about how to get the contents of a resource file that is loaded into Oracle as part of Java Stored Procedures package.<br/><br/>While working on enhancing some Java procedures, I added a simple Java logger class, fashioned after Log4J. To be able to dynamically configure the logging properties (log levels, threshold), I added in a properties file, bundled into the Jar file itself<strong><sup>1</sup></strong>. After loading the properties file into the Database (using <a title="LoadJava" href="http://docstore.mik.ua/orelly/oracle/guide8i/ch09_04.htm" target="_blank">LoadJava</a> command; loaded as a Java Resource), I wanted a way to "see" what's inside the file. That's when I posted a question on StackOverflow, but eventually googled and found the answer. See my question and my own answer <a title="Java Stored Procedure - Reading Resource File" href="http://stackoverflow.com/questions/9403164/java-stored-procedure-resource-files-in-oracle/9403199#9403199" target="_blank">here</a>. And here is the script, I came up with:<br/><pre>SET SERVEROUTPUT ON<br/>EXEC DBMS_JAVA.SET_OUTPUT (1000000);<br/>DECLARE<br/> bText <a title="CLOB" href="http://www.orafaq.com/wiki/CLOB" target="_blank">CLOB</a>; <br/> len PLS_INTEGER;<br/> offset PLS_INTEGER;<br/> text VARCHAR2(2000);<br/>BEGIN<br/><a title="lob CreateTemporary" href="http://www.toadworld.com/KNOWLEDGE/KnowledgeXpertforOracle/tabid/648/TopicID/DLPP/Default.aspx" target="_blank"><strong> DBMS_LOB</strong>.CreateTemporary</a>(bText, FALSE );<br/><a title="DBMS_JAVA" href="http://docs.oracle.com/cd/B19306_01/java.102/b14187/appendixa.htm" target="_blank"><strong> DBMS_JAVA</strong>.Export_Resource</a>('<strong>LAJavaSP.properties</strong>', bText);<br/> len := 1000; -- length of the lob to read.<br/> offset := 1;<br/><a title="DBMS_LOB.Read" href="http://www.toadworld.com/KNOWLEDGE/KnowledgeXpertforOracle/tabid/648/TopicID/DLPF/Default.aspx" target="_blank"> DBMS_LOB.Read</a>(bText, len, offset, text); <br/> <a title="Put_Line" href="http://thinkoracle.blogspot.com/2005/10/dbmsoutputputline.html" target="_blank">DBMS_OUTPUT.Put_Line</a>(text);<br/>END;<br/>/</pre><br/>This printed the following:<br/><pre><span style="color:#000000;">logLevel=ERROR<br/>dbLogLevel=ERROR</span></pre><br/><div></div><br/><div>For now, that's all I have in the properties file. Hoping to add more later.</div><br/><div></div><br/><br/><hr /><br/><br/><div><strong>1 </strong>Adding the properties file into the jar file makes it easy to read it using <code>getSystemResourceAsStream.</code> Otherwise, we will have to worry about Directory objects, Permissions etc.<strong><br/></strong></div>Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-32950477868802705502013-01-22T13:02:00.000-08:002013-06-20T22:02:31.592-07:00(Project) Withdrawal Symptoms<a href="http://www.toondoo.com/cartoon/5810039"><img title="Click to View Full Size Image" alt="Different withdrawal" src="http://static.toondoo.com/public/c/a/r/cartoonsv//toons/cool-cartoon-5810039.png" width="450" border="0" /></a><br/><div style="font-size:11px;font-family:Arial, Tahoma, Helvetica, sans-serif;text-align:left;width:100%;">By <a href="http://www.toondoo.com/user/cartoonsv">cartoonsv</a> | <a href="http://www.toondoo.com/cartoon/5810039">View this Toon at ToonDoo</a> | <a href="http://www.toondoo.com/">Create your own Toon</a></div><br/>I must admit. I think, I am going through withdrawal symptoms. No, not that kind. I am not an alcoholic, and I don't do drugs. I do drink coffee, but definitely no withdrawal from there! This is a different kind of withdrawal. I think, I just OD'd on work and now I am feeling the project withdrawal symptoms. I did a regular day's work today, it feels like I am not doing any real work. I get out of the building, while there is some light, it feels like I am leaving too early!<br/><br/>I just finished a project of refactoring one of our websites for performance. It took a lot of long days and nights, incessant reading, writing, thinking. Lot of headaches, anxieties, uncertainties. At times, it looked impossible, insurmountable and the team members expressed concerns, doubts. Even the week we were getting ready, there were issues, bugs, failed tests, network issues. After all, we are in the tax (peak) season already, is this a good time to deploy such a big change? Is it going to happen? Is it worth it? It was like writing/reading a mystery novel at the same time. What is the end? I am writing it, but I am also hoping to read it ahead. Couldn't stand the suspense no more.<br/><br/>The number of tasks grew and grew, as I picked up slack for team members that went on vacation. Finally, it's over!! poof! gone! We installed the system Thursday night. No issues!!!!! What? No way! I was looking for issues. But none. It was a good thing, lot of planning and work went into it, yet it felt like a very bland ending of the mystery novel. Phew! No issues! It feels good. But, why does it feel like nothing happened? The rest of the team members haven't said anything yet. Did the implementation really happen? I look at the beautiful log file created by Log4J smiling at me. The changes actually helped us catch an issue, we otherwise wouldn't. Yes, we did it!!<br/><br/>Now, I am catching up with the rest of life. I was here, wasn't I? Calls to be made, photos to share, trips to make, people to catch up with. In the past few months, I would come home late from work, eat and jump right back on the computer, pouring over the source code, popping in and out of forums, go round and round the world wide web, eyes glued to pages of the books and the mountain of pages I printed. Now, it feels like I don't have anything to do. But, it's a nice withdrawal! No hangovers, no issues! Work, life balance? My wife thinks, I am back to life!!! Good to be back!Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-20670144293677341072013-01-21T10:47:00.000-08:002013-06-20T22:02:31.589-07:00Software State of Affairs - Tools of TradeEarly man did a lot of things by hand, but soon started inventing <a href="http://www.ehow.com/info_8093762_tools-early-humans.html" target="_blank">tools</a> (<a title="Simple Machines" href="http://sln.fi.edu/qa97/spotlight3/spotlight3.html" target="_blank">Simple machines</a> in physics) to help him do things quicker and easier. Modern tools are way too complex than these early predecessors.<br/><br/><strong>Tools of Trade<sup>1</sup></strong><br/><br/>In the modern world, we depend on tools so much, we don't even realize. See this little <a href="http://edheads.org/activities/simple-machines/frame_loader.htm" target="_blank">Physics activity page</a> to see how many you may be using. Without tools, our lifestyles as we know will cease to exist. Every trade has associated tools, which we will call Tools of trade (In legalese it has a more restrictive meaning. See note 1). When we call a plumber, we expect him to come with the tools necessary, like the wrench etc. The electricians use different set of tools of which,<a href="http://home.howstuffworks.com/electrical-tools.htm" target="_blank"> voltage tester </a>may be the most important. If they don't have the right tools, we know what will happen. At the very least, you will be looking up Yellow Pages to call the next technician.<br/><table><br/><tbody><br/><tr><br/><td><a href="http://www.toondoo.com/cartoon/5802127"><img title="Click to View Full Size Image" alt="Lack of Tools" src="http://static.toondoo.com/public/c/a/r/cartoonsv//toons/cool-cartoon-5802127.png" border="0" /></a></td><br/><td><a href="http://www.toondoo.com/cartoon/5802145"><img title="Click to View Full Size Image" alt="wrong tool" src="http://static.toondoo.com/public/c/a/r/cartoonsv//toons/cool-cartoon-5802145.png" width="380" height="430" border="0" /></a></td><br/></tr><br/></tbody><br/></table><br/><strong>Software Tool Kit</strong><br/><br/>IT field borrows a lot of terminologies from other fields. We are engineers and technicians and architects that "build" software products, just that you won't be able to see the finished products except on your computer screen. While building, running, testing software products, we use a lot of tools. Like the product they help to build, these tools are software themselves. We use Compilers, Editors, Linkers, Runtime etc. to build and execute our software. But, unlike a plumber going to <a class="zem_slink" title="The Home Depot" href="http://maps.google.com/maps?ll=33.9065,-84.4872&spn=1.0,1.0&q=33.9065,-84.4872 (The%20Home%20Depot)&t=h" target="_blank" rel="geolocation">Home Depot</a> to get his tools, we could simply download it. At the same time, while a plumber's tool chest may always look alike, our tool chest varies depending on the field and the software we build. And unlike a plumber or an electrician, we have the luxury of building our own tools, more so now in the age of Open Source.<br/><br/><strong>Tools I use</strong><br/><br/>As a consultant, I am constantly exposed to different tools at different sites and for some tasks, I tend to keep my own time tested tools. Here are some Open Source tools, I currently use. Often times, I use more than one tool for a task and combine the results. These are mostly for Windows environment. I will share other tools and for Linux separately.<br/><br/><a href="http://sites.google.com/a/lacity.org/lataxdev/the-team" target="_blank">Windows Grep </a>I use it to search for text in files. Same like what Notepad++ or Textpad would search, but has a little more flexibility like the grep.<br/><br/>Many years ago, I used Examdiff extensively. Lately, <a href="http://www.google.com/url?q=http%3A%2F%2Fwinmerge.org%2F&sa=D&sntz=1&usg=AFrqEzcnWxW5k3bTURORYYXNQQqbjKbDgg" target="_blank">WinMerge</a> is the diff tool I use most frequently. WinMerge helps me compare at directory level and helps to merge files quickly. There are some limitations, but I love it. I also use K-Diff3 and CSDiff as each of these offer something unique. These are all free software. Another great diff tool I found was Beyond Compare. My next software purchase would be this. The above tools don't provide a good report for directory compare (Beyond Compare may). To do this, I found <a href="http://gnuwin32.sourceforge.net/packages/diffutils.htm">diffUtils</a> and a <a title="script to colorize diff output" href="http://www.linuxjournal.com/content/convert-diff-output-colorized-html" target="_blank">nice script</a> I found on the internet, to format the output in HTML.<br/><br/><a href="http://www.google.com/url?q=http%3A%2F%2Fgetgreenshot.org%2F&sa=D&sntz=1&usg=AFrqEzfimkeP0zP6Zqa1FyLNX91TXZp2Rw" target="_blank">Greenshot</a> This is the open source screen capture tool I use. Pikpik is a great tool as well. Again, each of these have some feature that others lack.<br/><br/><a href="http://www.google.com/url?q=http%3A%2F%2Fwww.gabrieleponti.com%2Fsoftware%2F&sa=D&sntz=1&usg=AFrqEzd2FwegE6BebWdtGT-v_Yq--lqvQw" target="_blank">OpenCommandPromptHere</a> There are several names like Command Prompt Here, SendtoToys etc. Essentially, creates a context sensitive menu in Windows Explorer, so you can open command prompt from where ever you are in Explorer. (you don't have to CD into any directory manually).<br/><br/><a href="http://www.google.com/url?q=http%3A%2F%2Fwww.baremetalsoft.com%2Fbaretail%2F&sa=D&sntz=1&usg=AFrqEzcoZA_dr99Gn91yz2oWpqhRHdlnwA" target="_blank">BareTail</a> I recommended this earlier; I use this to tail Jaguar.log etc. It's really good. You can highlight lines with different phrases.<br/><br/><a href="http://www.google.com/url?q=http%3A%2F%2Fwww.yworks.com%2Fen%2Fproducts_yed_about.html&sa=D&sntz=1&usg=AFrqEzcMwczmuEVi3KREMJdekzoHR6iKGg" target="_blank">YEd</a> This I use a lot to do diagramming. It's more a graph editor than a Graphics editor. Works for me. While researching on diagramming, I found UMLet to be interesting. Dot, Dotty and GraphViz help me with automatically generating some flow diagrams.<br/><br/>Apart from these I use Ant, Eclipse, JMeter, Xampp etc as part of the my development environment. I also use Libre Office (Open Office's successor) a lot.<br/><br/><strong>Tools I build</strong><br/><br/>I am always looking for ways to make the computer do the work for me, instead of doing things manually. I feel, if a task has to be done more than once, it deserves a script or a tool. When the tools available are not good enough for a particular task, I try to build some on my own as well. But this is also an excellent opportunity to learn new techniques and keep your skills sharp. I do these during down times and/or as part of the main task.<br/><br/>I am sharing some of my tool building experiences here, hoping it would benefit someone else. If you are a Software developer and somewhat less motivated with the stuff you are working on, give tool building a try next time. The software will benefit from it too. All you need is some imagination and some laziness to resist doing anything manually! Gone are the days, when you had to purchase special software to build such tools. You will be amazed to see how much programming capabilities a standard computer can offer you. For e.g., if you have Word or Excel, you can program build powerful macros (VBA programs). Apart from these, you always have Open Source scripting languages like PERL, Python, PHP etc.<br/><br/>If you recognize early enough, the tools you build could become integral part of a project as well. At Velos, while working on the task of upgrading the database (Sybase SQL Anywhere) for a newer specification, I decided convert the specification into an Access Database. Once the new Spec. was in a database, I was able to compare existing database elements with the new ones and voilà!, the database upgrade script could be generated right out of the database itself. They had a Powerbuilder application to design and save screens (datawindows) in database (blob columns). After going through problems with the blob, decided to write a tool to view, edit, correct screens outside of the application. Few more times of using this, I ended up in writing a simple scripting language to perform these same tasks in a batch mode. Necessity is the mother of invention! Several of these tools were actually used by the support person to identify and fix issues at customer sites.<br/><br/>Some of the casual tools I build for one time use, end up as permanent additions to the team's tool chest. At Capital group, one developer was collecting batch job stats on a daily basis, and when he went on vacation, he gave a page long instructions on how to get these details in Excel. The task was, to go to the web pages (someone with tool mentality, pulled the stats from <a title="Autosys" href="http://viralpatel.net/blogs/autosys-create-manage-jobs-autosys-scheduler-job-monitoring-autosys/" target="_blank">Autosys</a> into the page, using CGI-PERL), copy the job names to a spreadsheet and make adjustments to be presentable. When my turn came, I got tired of copying, pasting and correcting pretty quickly. Before the next time, I added a "<a title="Excel Web Query" href="http://www.techrepublic.com/article/pull-data-into-microsoft-excel-with-web-queries/6115870" target="_blank">Web Query</a>" in Excel to pull up the web pages directly into the spreadsheet and wrote some VBA macros to clean up and present the spreadsheets. Eventually, it became full-fledged product in it's own right, everyone in the team started using it. Page long instructions gone. (Of course, when I left, I left a few pages long document about the macro, but that's another story).<br/><br/>Similarly, in the same company there was another task, developers and/or the analysts had to do. Every now and then company received fund codes from outside (at the time I didn't know much, these are the <a href="www.library.cornell.edu/abld/abld03/sicnaicgic.ppt" target="_blank">GICS</a> codes we received from <a title="History of MSCI" href="http://internationalinvest.about.com/od/globalmarkets101/a/mscihistory.htm" target="_blank">MSCI</a>), that had to be reconciled against codes that existed in the system already. Some codes may be replaced with new ones, some may get new descriptions, some may be discontinued. So, they would enter these "changes" in a spreadsheet or word document and then ask a developer to convert these to SQL (INSERT/UPDATE/DELETE) statements to be put into production. Enter yours truly, the lazy user. I wrote some excel macros that would import the data from the Oracle table (using ODBC), reconcile and generate the SQLs for the developer to run in production. Yes, it did take a little more time to develop and test it, but next time similar request came, it was a breeze.<br/><a href="http://power2build.files.wordpress.com/2013/01/screenshot-from-2013-01-21-165131.png" target="_blank"><img class="alignleft" alt="Query Tool" src="http://power2build.files.wordpress.com/2013/01/screenshot-from-2013-01-21-165131.png" width="143" height="188" /></a><br/><br/>An Excel macro dubbed "Query Tool", I developed at Capital Group tops it all. It again started as a manual task. Goal was to identify some codes (for e.g., security codes) from another system and see if it existed in our database. This was done manually too. User would extract data from the table into a spreadsheet and look up each code. Sometimes, codes might have been mistyped, so they have to match partially or look at other attributes to identify and correct. At the end of this, they may have gone through a spreadsheet with 1000's of codes in several passes, identifying codes from another list they had. Time consuming, laborious and error prone. If this is not what the computer does best, what else? So, I started coding the excel macros to do the repeated search of the codes they entered. Little by little, it became a full fledged querying tool (using Database Query - through ODBC - in Excel). The tab containing the actual SQLs helped customized the lists further. What resulted was a nice product. The users loved it.<br/><br/>These one-off solutions may also help you save a project. At <a title="Federal Home Loan Bank, San Francisco" href="www.fhlbsf.com" target="_blank">FHLB</a>, we were working on a data extraction project. This was part of the data reporting requirements of the bank set by <a title="Federal Housing Finance Board" href="www.fhfa.gov/" target="_blank">FHFB</a>. I created an Access Database as a prototype to a much bigger data-warehouse built in Oracle by a team of ETL developers. I kept this as a tool to validate the data/report from the ETL tools, before we sent the data to <a title="Federal Housing Finance Board" href="www.fhfa.gov/" target="_blank">FHFB</a>. All was going well, but suddenly one day, we found a flaw in the Data-warehouse design and the multi-million dollar project simply collapsed. Luckily my boss saw this risk early, supported my efforts to spruce up my Access Database. Believe it or not, my little tool eventually became the official "data reporting tool" for the company, at least the first year.<br/><br/><strong>Journey Continues..</strong><br/><br/>Recently, one developer mentioned about how to check a file in Textpad for a record with odd length. This was the idea behind this <a title="File Checking – Perl to the rescue" href="http://power2build.wordpress.com/2011/12/11/262/">PERL script</a>. Another opportunity for such a script was where a batch program creates a large file (XML in this case) that had to be split into smaller chunks. Perfect candidate again for a PERL or a Python script. I did a quick prototype, but it died down for lack of interest. I hear they do it with some tools like access DB and a lot of manual "pulling the hair". And then there are tasks like parsing logs and so forth. A well written script can analyze and report stats from these files, in a jiffy.<br/><br/>The other day, we were discussing testing for a Database upgrade. I brought up the topic of monitoring and my boss said, "you can make yet another tool for it"! I know he was kidding, but I probably will or if I am in luck, I may be able to just download something from the internet. The journey continues!<br/><br/><hr /><br/><br/><strong>1. </strong>Tools of Trade has a different meaning in legalese. It's the tools your livelihood depends on, so in case of bankruptcy, they will not be taken away by creditors.Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0tag:blogger.com,1999:blog-785353890590284114.post-82961556419630223722013-01-20T15:43:00.000-08:002013-06-20T22:02:31.609-07:00Software State of Affairs - Technical Status QuoI just finished a project to refactor our Web application, written in JSP many years ago. This runs in our EA Server (Ver 5.5) environment. The environment itself is 8+ years old, and the versions are reflective of that time. The technology used was already outdated by the time the software went into production. There were several violations of the modern web programming techniques. Lot of embedded Java coding, at will call/connections to servers and database etc. caused the program to fail during peak loads.<br/><br/>Well, there is a lot to talk about that software and the improvements we did. I will do so in other posts. In this post, I want to focus on the issue I see in every <a class="zem_slink" title="Software maintenance" href="http://en.wikipedia.org/wiki/Software_maintenance" target="_blank" rel="wikipedia">software maintenance</a> projects - the status quo in keeping the software architecture intact, irrespective how much it changes in business functionality.<br/><br/>I frequently compare software to a car. The comparison arises in that both are built, run, need to be maintained, and tuned up. That's where the comparison ends. We don't keep adding functionalities to a car. We add bells and whistles like tint, color, alarm etc, but never add functionality or change it's function completely; we don't make a car run on square wheels or sideways. Though the engine gets hot, we don't use a it as a toaster or to cook chicken. But, in software we do exactly that.<br/><a href="http://www.toondoo.com/cartoon/5803833"><img title="Click to View Full Size Image" alt="computer make coffee" src="http://static.toondoo.com/public/c/a/r/cartoonsv//toons/cool-cartoon-5803833.png" border="0" /></a><br/><br/>When a custom software is designed or purchased, the users typically get all the functions they need. But, over the years users change and so does the business. We keep adding enhancements, <a class="zem_slink" title="Patch management" href="http://www.symantec.com/patch-management" target="_blank" rel="symantec">bug fixes</a> to the software constantly to meet the (sometimes unreasonable) business requirements that were never dreamed of, when we built the original software. The software thus gets bloated over a time period. We don't adjust the technical architecture or design to match this new software obesity and it eventually comes to a crawl, at which point everyone blames the software and want to replace it. By then, it may be too late! Many custom software out there have nice front ends, but - pardon my French - crappy code inside. It's like a flash new bus pulled by a mule, no offence to the mule!! I blame this on Software Status Quo! We tend to cater to business so much, we neglect the software itself in the process!!<br/><br/><a href="http://www.toondoo.com/cartoon/5803934"><img title="Click to View Full Size Image" alt="mule inside" src="http://static.toondoo.com/public/c/a/r/cartoonsv//toons/cool-cartoon-5803934.png" border="0" /></a><br/><br/>Lot of this is due to the way companies budget for Software projects. Several departments tend to have a "lights-on" policy, which means keeping the software up and running 24X7 without any outage. Companies typically have a lot of budget for "operations" but not real software maintenance. As a consultant, I've seen IT departments struggle to define my work as part of "lights on", so they could continue to get funds for my contract.<br/><br/>Typically, the business side budget pays for the "operations". Except for a handful of savvy business managers, rest of them don't really understand software. To them, the "pretty" screens and "nice" reports are the software. As long as these happen, software is working fine. When you work with such managers, it is not easy to get budget for retrofitting software. "Business" requirements will always take precedence to technical requirements.<br/><br/>But, business is not entirely to blame for this. Recently, when I started introducing the changes in our software, I saw advantages of using more Java in a mostly Powerbuilder environment. There was some awe initially, then reluctance and finally an indifferent acceptance. New architecture means, new problems and tougher maintenance, at least initially. Now, I see why this status quo is continuing. I always hear senior programmers say, if it's not broken and/or users didn't want it, we wouldn't fix it. Where as, this is a safe approach to keep business happy, it hurts the software in the long run.<br/><br/>To end such status quo, business must stop treating software like stationary and recognize and allocate budget for it to be "tuned" regularly. I also feel, real end users (not just business managers) must be regularly consulted as to the performance and usage of the application. I've seen many a software projects fail, in spite of extensive design and funding, because end users simply rejected the software. Developers must also constantly look for ways to change and improve the software itself (not just it's business functionality) and strive to include these in their releases. I see several developers less motivated with the routine business fixes. Adding in technical items, may help to re-energize the developer base.<br/><br/>Many years ago, I worked in a warehouse in NJ where they were using an AS/400 for order tracking, but the software itself was so outdated, the users were improvising. Believe it or not, they would print a report out of this system, then they would input that information into System/36 along with accounting information and export to a file. The next smart user takes that to mainframe to finish up the final entry, where corporate wanted the original shipping and accounts information. Those systems people in Corporate, 3000 miles away conveniently ignored this status quo, that this process actually continued for many years until truly yours stepped in! Ironically, around the same time, corporate headquarters poured in millions to port some of their mainframe applications to flashy client/server tools.<br/><br/>I normally keep notes of all technical issues or a wish list for the software I work on. Then when I get to work on some business requirement in that software, I include any appropriate technical fix. Recently, while working on a business requirement involving Java stored procedures, I rewrote the logging mechanism completely, so we can support it better. Previously, developers were compiling and distributing this code to DBAs completely outside of the build process. If one class had to be updated, just compile it and patch it into the Jar file!!! This was error prone and I found some source code older than compiled code. I introduced eclipse and Ant to reduce human errors. Of course, it took longer to implement the task, but it pays for itself in the long run!<br/><br/>Oh yeah, we did solve the impasse at the warehouse in NJ, but it needed some thinking outside the box. I got the blessing from the local business director and with the help from users and corporate developers, we rolled the users to corporate mainframe (of course, after making necessary changes on the mainframe to accommodate them) and wheeled the AS/400 out, the one I was there to support in the first place! The company saved a bundle and everyone was happy!Anonymoushttp://www.blogger.com/profile/11455363550386831996noreply@blogger.com0