This is from 2010, but interesting read
http://www.sciencedaily.com/releases/2010/03/100325100848.htm
Showing posts with label Misc. Show all posts
Showing posts with label Misc. Show all posts
Tuesday, January 22, 2013
2012 in review - Auto generated Wordpress Report
The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.
Here's an excerpt:
Click here to see the complete report.
SamV: Wasn't exactly a good year, but this is the year, this blog is turning around. I am finally seeing some visitors. I am still in the process of improving this. If you happened to stop by here and have some ideas as to how to improve the user experience, please feel free add as comments. I appreciate it. Have a wonderful 2013!!!
Here's an excerpt:
600 people reached the top of Mt. Everest in 2012. This blog got about 4,900 views in 2012. If every person who reached the top of Mt. Everest viewed this blog, it would have taken 8 years to get that many views.
Click here to see the complete report.
SamV: Wasn't exactly a good year, but this is the year, this blog is turning around. I am finally seeing some visitors. I am still in the process of improving this. If you happened to stop by here and have some ideas as to how to improve the user experience, please feel free add as comments. I appreciate it. Have a wonderful 2013!!!
(Project) Withdrawal Symptoms
I must admit. I think, I am going through withdrawal symptoms. No, not that kind. I am not an alcoholic, and I don't do drugs. I do drink coffee, but definitely no withdrawal from there! This is a different kind of withdrawal. I think, I just OD'd on work and now I am feeling the project withdrawal symptoms. I did a regular day's work today, it feels like I am not doing any real work. I get out of the building, while there is some light, it feels like I am leaving too early!
I just finished a project of refactoring one of our websites for performance. It took a lot of long days and nights, incessant reading, writing, thinking. Lot of headaches, anxieties, uncertainties. At times, it looked impossible, insurmountable and the team members expressed concerns, doubts. Even the week we were getting ready, there were issues, bugs, failed tests, network issues. After all, we are in the tax (peak) season already, is this a good time to deploy such a big change? Is it going to happen? Is it worth it? It was like writing/reading a mystery novel at the same time. What is the end? I am writing it, but I am also hoping to read it ahead. Couldn't stand the suspense no more.
The number of tasks grew and grew, as I picked up slack for team members that went on vacation. Finally, it's over!! poof! gone! We installed the system Thursday night. No issues!!!!! What? No way! I was looking for issues. But none. It was a good thing, lot of planning and work went into it, yet it felt like a very bland ending of the mystery novel. Phew! No issues! It feels good. But, why does it feel like nothing happened? The rest of the team members haven't said anything yet. Did the implementation really happen? I look at the beautiful log file created by Log4J smiling at me. The changes actually helped us catch an issue, we otherwise wouldn't. Yes, we did it!!
Now, I am catching up with the rest of life. I was here, wasn't I? Calls to be made, photos to share, trips to make, people to catch up with. In the past few months, I would come home late from work, eat and jump right back on the computer, pouring over the source code, popping in and out of forums, go round and round the world wide web, eyes glued to pages of the books and the mountain of pages I printed. Now, it feels like I don't have anything to do. But, it's a nice withdrawal! No hangovers, no issues! Work, life balance? My wife thinks, I am back to life!!! Good to be back!
Monday, January 21, 2013
Software State of Affairs - Tools of Trade
Early man did a lot of things by hand, but soon started inventing tools (Simple machines in physics) to help him do things quicker and easier. Modern tools are way too complex than these early predecessors.
Tools of Trade1
In the modern world, we depend on tools so much, we don't even realize. See this little Physics activity page to see how many you may be using. Without tools, our lifestyles as we know will cease to exist. Every trade has associated tools, which we will call Tools of trade (In legalese it has a more restrictive meaning. See note 1). When we call a plumber, we expect him to come with the tools necessary, like the wrench etc. The electricians use different set of tools of which, voltage tester may be the most important. If they don't have the right tools, we know what will happen. At the very least, you will be looking up Yellow Pages to call the next technician.
Software Tool Kit
IT field borrows a lot of terminologies from other fields. We are engineers and technicians and architects that "build" software products, just that you won't be able to see the finished products except on your computer screen. While building, running, testing software products, we use a lot of tools. Like the product they help to build, these tools are software themselves. We use Compilers, Editors, Linkers, Runtime etc. to build and execute our software. But, unlike a plumber going to Home Depot to get his tools, we could simply download it. At the same time, while a plumber's tool chest may always look alike, our tool chest varies depending on the field and the software we build. And unlike a plumber or an electrician, we have the luxury of building our own tools, more so now in the age of Open Source.
Tools I use
As a consultant, I am constantly exposed to different tools at different sites and for some tasks, I tend to keep my own time tested tools. Here are some Open Source tools, I currently use. Often times, I use more than one tool for a task and combine the results. These are mostly for Windows environment. I will share other tools and for Linux separately.
Windows Grep I use it to search for text in files. Same like what Notepad++ or Textpad would search, but has a little more flexibility like the grep.
Many years ago, I used Examdiff extensively. Lately, WinMerge is the diff tool I use most frequently. WinMerge helps me compare at directory level and helps to merge files quickly. There are some limitations, but I love it. I also use K-Diff3 and CSDiff as each of these offer something unique. These are all free software. Another great diff tool I found was Beyond Compare. My next software purchase would be this. The above tools don't provide a good report for directory compare (Beyond Compare may). To do this, I found diffUtils and a nice script I found on the internet, to format the output in HTML.
Greenshot This is the open source screen capture tool I use. Pikpik is a great tool as well. Again, each of these have some feature that others lack.
OpenCommandPromptHere There are several names like Command Prompt Here, SendtoToys etc. Essentially, creates a context sensitive menu in Windows Explorer, so you can open command prompt from where ever you are in Explorer. (you don't have to CD into any directory manually).
BareTail I recommended this earlier; I use this to tail Jaguar.log etc. It's really good. You can highlight lines with different phrases.
YEd This I use a lot to do diagramming. It's more a graph editor than a Graphics editor. Works for me. While researching on diagramming, I found UMLet to be interesting. Dot, Dotty and GraphViz help me with automatically generating some flow diagrams.
Apart from these I use Ant, Eclipse, JMeter, Xampp etc as part of the my development environment. I also use Libre Office (Open Office's successor) a lot.
Tools I build
I am always looking for ways to make the computer do the work for me, instead of doing things manually. I feel, if a task has to be done more than once, it deserves a script or a tool. When the tools available are not good enough for a particular task, I try to build some on my own as well. But this is also an excellent opportunity to learn new techniques and keep your skills sharp. I do these during down times and/or as part of the main task.
I am sharing some of my tool building experiences here, hoping it would benefit someone else. If you are a Software developer and somewhat less motivated with the stuff you are working on, give tool building a try next time. The software will benefit from it too. All you need is some imagination and some laziness to resist doing anything manually! Gone are the days, when you had to purchase special software to build such tools. You will be amazed to see how much programming capabilities a standard computer can offer you. For e.g., if you have Word or Excel, you can program build powerful macros (VBA programs). Apart from these, you always have Open Source scripting languages like PERL, Python, PHP etc.
If you recognize early enough, the tools you build could become integral part of a project as well. At Velos, while working on the task of upgrading the database (Sybase SQL Anywhere) for a newer specification, I decided convert the specification into an Access Database. Once the new Spec. was in a database, I was able to compare existing database elements with the new ones and voilà!, the database upgrade script could be generated right out of the database itself. They had a Powerbuilder application to design and save screens (datawindows) in database (blob columns). After going through problems with the blob, decided to write a tool to view, edit, correct screens outside of the application. Few more times of using this, I ended up in writing a simple scripting language to perform these same tasks in a batch mode. Necessity is the mother of invention! Several of these tools were actually used by the support person to identify and fix issues at customer sites.
Some of the casual tools I build for one time use, end up as permanent additions to the team's tool chest. At Capital group, one developer was collecting batch job stats on a daily basis, and when he went on vacation, he gave a page long instructions on how to get these details in Excel. The task was, to go to the web pages (someone with tool mentality, pulled the stats from Autosys into the page, using CGI-PERL), copy the job names to a spreadsheet and make adjustments to be presentable. When my turn came, I got tired of copying, pasting and correcting pretty quickly. Before the next time, I added a "Web Query" in Excel to pull up the web pages directly into the spreadsheet and wrote some VBA macros to clean up and present the spreadsheets. Eventually, it became full-fledged product in it's own right, everyone in the team started using it. Page long instructions gone. (Of course, when I left, I left a few pages long document about the macro, but that's another story).
Similarly, in the same company there was another task, developers and/or the analysts had to do. Every now and then company received fund codes from outside (at the time I didn't know much, these are the GICS codes we received from MSCI), that had to be reconciled against codes that existed in the system already. Some codes may be replaced with new ones, some may get new descriptions, some may be discontinued. So, they would enter these "changes" in a spreadsheet or word document and then ask a developer to convert these to SQL (INSERT/UPDATE/DELETE) statements to be put into production. Enter yours truly, the lazy user. I wrote some excel macros that would import the data from the Oracle table (using ODBC), reconcile and generate the SQLs for the developer to run in production. Yes, it did take a little more time to develop and test it, but next time similar request came, it was a breeze.
An Excel macro dubbed "Query Tool", I developed at Capital Group tops it all. It again started as a manual task. Goal was to identify some codes (for e.g., security codes) from another system and see if it existed in our database. This was done manually too. User would extract data from the table into a spreadsheet and look up each code. Sometimes, codes might have been mistyped, so they have to match partially or look at other attributes to identify and correct. At the end of this, they may have gone through a spreadsheet with 1000's of codes in several passes, identifying codes from another list they had. Time consuming, laborious and error prone. If this is not what the computer does best, what else? So, I started coding the excel macros to do the repeated search of the codes they entered. Little by little, it became a full fledged querying tool (using Database Query - through ODBC - in Excel). The tab containing the actual SQLs helped customized the lists further. What resulted was a nice product. The users loved it.
These one-off solutions may also help you save a project. At FHLB, we were working on a data extraction project. This was part of the data reporting requirements of the bank set by FHFB. I created an Access Database as a prototype to a much bigger data-warehouse built in Oracle by a team of ETL developers. I kept this as a tool to validate the data/report from the ETL tools, before we sent the data to FHFB. All was going well, but suddenly one day, we found a flaw in the Data-warehouse design and the multi-million dollar project simply collapsed. Luckily my boss saw this risk early, supported my efforts to spruce up my Access Database. Believe it or not, my little tool eventually became the official "data reporting tool" for the company, at least the first year.
Journey Continues..
Recently, one developer mentioned about how to check a file in Textpad for a record with odd length. This was the idea behind this PERL script. Another opportunity for such a script was where a batch program creates a large file (XML in this case) that had to be split into smaller chunks. Perfect candidate again for a PERL or a Python script. I did a quick prototype, but it died down for lack of interest. I hear they do it with some tools like access DB and a lot of manual "pulling the hair". And then there are tasks like parsing logs and so forth. A well written script can analyze and report stats from these files, in a jiffy.
The other day, we were discussing testing for a Database upgrade. I brought up the topic of monitoring and my boss said, "you can make yet another tool for it"! I know he was kidding, but I probably will or if I am in luck, I may be able to just download something from the internet. The journey continues!
1. Tools of Trade has a different meaning in legalese. It's the tools your livelihood depends on, so in case of bankruptcy, they will not be taken away by creditors.
Tools of Trade1
In the modern world, we depend on tools so much, we don't even realize. See this little Physics activity page to see how many you may be using. Without tools, our lifestyles as we know will cease to exist. Every trade has associated tools, which we will call Tools of trade (In legalese it has a more restrictive meaning. See note 1). When we call a plumber, we expect him to come with the tools necessary, like the wrench etc. The electricians use different set of tools of which, voltage tester may be the most important. If they don't have the right tools, we know what will happen. At the very least, you will be looking up Yellow Pages to call the next technician.
 |  |
Software Tool Kit
IT field borrows a lot of terminologies from other fields. We are engineers and technicians and architects that "build" software products, just that you won't be able to see the finished products except on your computer screen. While building, running, testing software products, we use a lot of tools. Like the product they help to build, these tools are software themselves. We use Compilers, Editors, Linkers, Runtime etc. to build and execute our software. But, unlike a plumber going to Home Depot to get his tools, we could simply download it. At the same time, while a plumber's tool chest may always look alike, our tool chest varies depending on the field and the software we build. And unlike a plumber or an electrician, we have the luxury of building our own tools, more so now in the age of Open Source.
Tools I use
As a consultant, I am constantly exposed to different tools at different sites and for some tasks, I tend to keep my own time tested tools. Here are some Open Source tools, I currently use. Often times, I use more than one tool for a task and combine the results. These are mostly for Windows environment. I will share other tools and for Linux separately.
Windows Grep I use it to search for text in files. Same like what Notepad++ or Textpad would search, but has a little more flexibility like the grep.
Many years ago, I used Examdiff extensively. Lately, WinMerge is the diff tool I use most frequently. WinMerge helps me compare at directory level and helps to merge files quickly. There are some limitations, but I love it. I also use K-Diff3 and CSDiff as each of these offer something unique. These are all free software. Another great diff tool I found was Beyond Compare. My next software purchase would be this. The above tools don't provide a good report for directory compare (Beyond Compare may). To do this, I found diffUtils and a nice script I found on the internet, to format the output in HTML.
Greenshot This is the open source screen capture tool I use. Pikpik is a great tool as well. Again, each of these have some feature that others lack.
OpenCommandPromptHere There are several names like Command Prompt Here, SendtoToys etc. Essentially, creates a context sensitive menu in Windows Explorer, so you can open command prompt from where ever you are in Explorer. (you don't have to CD into any directory manually).
BareTail I recommended this earlier; I use this to tail Jaguar.log etc. It's really good. You can highlight lines with different phrases.
YEd This I use a lot to do diagramming. It's more a graph editor than a Graphics editor. Works for me. While researching on diagramming, I found UMLet to be interesting. Dot, Dotty and GraphViz help me with automatically generating some flow diagrams.
Apart from these I use Ant, Eclipse, JMeter, Xampp etc as part of the my development environment. I also use Libre Office (Open Office's successor) a lot.
Tools I build
I am always looking for ways to make the computer do the work for me, instead of doing things manually. I feel, if a task has to be done more than once, it deserves a script or a tool. When the tools available are not good enough for a particular task, I try to build some on my own as well. But this is also an excellent opportunity to learn new techniques and keep your skills sharp. I do these during down times and/or as part of the main task.
I am sharing some of my tool building experiences here, hoping it would benefit someone else. If you are a Software developer and somewhat less motivated with the stuff you are working on, give tool building a try next time. The software will benefit from it too. All you need is some imagination and some laziness to resist doing anything manually! Gone are the days, when you had to purchase special software to build such tools. You will be amazed to see how much programming capabilities a standard computer can offer you. For e.g., if you have Word or Excel, you can program build powerful macros (VBA programs). Apart from these, you always have Open Source scripting languages like PERL, Python, PHP etc.
If you recognize early enough, the tools you build could become integral part of a project as well. At Velos, while working on the task of upgrading the database (Sybase SQL Anywhere) for a newer specification, I decided convert the specification into an Access Database. Once the new Spec. was in a database, I was able to compare existing database elements with the new ones and voilà!, the database upgrade script could be generated right out of the database itself. They had a Powerbuilder application to design and save screens (datawindows) in database (blob columns). After going through problems with the blob, decided to write a tool to view, edit, correct screens outside of the application. Few more times of using this, I ended up in writing a simple scripting language to perform these same tasks in a batch mode. Necessity is the mother of invention! Several of these tools were actually used by the support person to identify and fix issues at customer sites.
Some of the casual tools I build for one time use, end up as permanent additions to the team's tool chest. At Capital group, one developer was collecting batch job stats on a daily basis, and when he went on vacation, he gave a page long instructions on how to get these details in Excel. The task was, to go to the web pages (someone with tool mentality, pulled the stats from Autosys into the page, using CGI-PERL), copy the job names to a spreadsheet and make adjustments to be presentable. When my turn came, I got tired of copying, pasting and correcting pretty quickly. Before the next time, I added a "Web Query" in Excel to pull up the web pages directly into the spreadsheet and wrote some VBA macros to clean up and present the spreadsheets. Eventually, it became full-fledged product in it's own right, everyone in the team started using it. Page long instructions gone. (Of course, when I left, I left a few pages long document about the macro, but that's another story).
Similarly, in the same company there was another task, developers and/or the analysts had to do. Every now and then company received fund codes from outside (at the time I didn't know much, these are the GICS codes we received from MSCI), that had to be reconciled against codes that existed in the system already. Some codes may be replaced with new ones, some may get new descriptions, some may be discontinued. So, they would enter these "changes" in a spreadsheet or word document and then ask a developer to convert these to SQL (INSERT/UPDATE/DELETE) statements to be put into production. Enter yours truly, the lazy user. I wrote some excel macros that would import the data from the Oracle table (using ODBC), reconcile and generate the SQLs for the developer to run in production. Yes, it did take a little more time to develop and test it, but next time similar request came, it was a breeze.
An Excel macro dubbed "Query Tool", I developed at Capital Group tops it all. It again started as a manual task. Goal was to identify some codes (for e.g., security codes) from another system and see if it existed in our database. This was done manually too. User would extract data from the table into a spreadsheet and look up each code. Sometimes, codes might have been mistyped, so they have to match partially or look at other attributes to identify and correct. At the end of this, they may have gone through a spreadsheet with 1000's of codes in several passes, identifying codes from another list they had. Time consuming, laborious and error prone. If this is not what the computer does best, what else? So, I started coding the excel macros to do the repeated search of the codes they entered. Little by little, it became a full fledged querying tool (using Database Query - through ODBC - in Excel). The tab containing the actual SQLs helped customized the lists further. What resulted was a nice product. The users loved it.
These one-off solutions may also help you save a project. At FHLB, we were working on a data extraction project. This was part of the data reporting requirements of the bank set by FHFB. I created an Access Database as a prototype to a much bigger data-warehouse built in Oracle by a team of ETL developers. I kept this as a tool to validate the data/report from the ETL tools, before we sent the data to FHFB. All was going well, but suddenly one day, we found a flaw in the Data-warehouse design and the multi-million dollar project simply collapsed. Luckily my boss saw this risk early, supported my efforts to spruce up my Access Database. Believe it or not, my little tool eventually became the official "data reporting tool" for the company, at least the first year.
Journey Continues..
Recently, one developer mentioned about how to check a file in Textpad for a record with odd length. This was the idea behind this PERL script. Another opportunity for such a script was where a batch program creates a large file (XML in this case) that had to be split into smaller chunks. Perfect candidate again for a PERL or a Python script. I did a quick prototype, but it died down for lack of interest. I hear they do it with some tools like access DB and a lot of manual "pulling the hair". And then there are tasks like parsing logs and so forth. A well written script can analyze and report stats from these files, in a jiffy.
The other day, we were discussing testing for a Database upgrade. I brought up the topic of monitoring and my boss said, "you can make yet another tool for it"! I know he was kidding, but I probably will or if I am in luck, I may be able to just download something from the internet. The journey continues!
1. Tools of Trade has a different meaning in legalese. It's the tools your livelihood depends on, so in case of bankruptcy, they will not be taken away by creditors.
Sunday, January 20, 2013
Software State of Affairs - Technical Status Quo
I just finished a project to refactor our Web application, written in JSP many years ago. This runs in our EA Server (Ver 5.5) environment. The environment itself is 8+ years old, and the versions are reflective of that time. The technology used was already outdated by the time the software went into production. There were several violations of the modern web programming techniques. Lot of embedded Java coding, at will call/connections to servers and database etc. caused the program to fail during peak loads.
Well, there is a lot to talk about that software and the improvements we did. I will do so in other posts. In this post, I want to focus on the issue I see in every software maintenance projects - the status quo in keeping the software architecture intact, irrespective how much it changes in business functionality.
I frequently compare software to a car. The comparison arises in that both are built, run, need to be maintained, and tuned up. That's where the comparison ends. We don't keep adding functionalities to a car. We add bells and whistles like tint, color, alarm etc, but never add functionality or change it's function completely; we don't make a car run on square wheels or sideways. Though the engine gets hot, we don't use a it as a toaster or to cook chicken. But, in software we do exactly that.
When a custom software is designed or purchased, the users typically get all the functions they need. But, over the years users change and so does the business. We keep adding enhancements, bug fixes to the software constantly to meet the (sometimes unreasonable) business requirements that were never dreamed of, when we built the original software. The software thus gets bloated over a time period. We don't adjust the technical architecture or design to match this new software obesity and it eventually comes to a crawl, at which point everyone blames the software and want to replace it. By then, it may be too late! Many custom software out there have nice front ends, but - pardon my French - crappy code inside. It's like a flash new bus pulled by a mule, no offence to the mule!! I blame this on Software Status Quo! We tend to cater to business so much, we neglect the software itself in the process!!
Lot of this is due to the way companies budget for Software projects. Several departments tend to have a "lights-on" policy, which means keeping the software up and running 24X7 without any outage. Companies typically have a lot of budget for "operations" but not real software maintenance. As a consultant, I've seen IT departments struggle to define my work as part of "lights on", so they could continue to get funds for my contract.
Typically, the business side budget pays for the "operations". Except for a handful of savvy business managers, rest of them don't really understand software. To them, the "pretty" screens and "nice" reports are the software. As long as these happen, software is working fine. When you work with such managers, it is not easy to get budget for retrofitting software. "Business" requirements will always take precedence to technical requirements.
But, business is not entirely to blame for this. Recently, when I started introducing the changes in our software, I saw advantages of using more Java in a mostly Powerbuilder environment. There was some awe initially, then reluctance and finally an indifferent acceptance. New architecture means, new problems and tougher maintenance, at least initially. Now, I see why this status quo is continuing. I always hear senior programmers say, if it's not broken and/or users didn't want it, we wouldn't fix it. Where as, this is a safe approach to keep business happy, it hurts the software in the long run.
To end such status quo, business must stop treating software like stationary and recognize and allocate budget for it to be "tuned" regularly. I also feel, real end users (not just business managers) must be regularly consulted as to the performance and usage of the application. I've seen many a software projects fail, in spite of extensive design and funding, because end users simply rejected the software. Developers must also constantly look for ways to change and improve the software itself (not just it's business functionality) and strive to include these in their releases. I see several developers less motivated with the routine business fixes. Adding in technical items, may help to re-energize the developer base.
Many years ago, I worked in a warehouse in NJ where they were using an AS/400 for order tracking, but the software itself was so outdated, the users were improvising. Believe it or not, they would print a report out of this system, then they would input that information into System/36 along with accounting information and export to a file. The next smart user takes that to mainframe to finish up the final entry, where corporate wanted the original shipping and accounts information. Those systems people in Corporate, 3000 miles away conveniently ignored this status quo, that this process actually continued for many years until truly yours stepped in! Ironically, around the same time, corporate headquarters poured in millions to port some of their mainframe applications to flashy client/server tools.
I normally keep notes of all technical issues or a wish list for the software I work on. Then when I get to work on some business requirement in that software, I include any appropriate technical fix. Recently, while working on a business requirement involving Java stored procedures, I rewrote the logging mechanism completely, so we can support it better. Previously, developers were compiling and distributing this code to DBAs completely outside of the build process. If one class had to be updated, just compile it and patch it into the Jar file!!! This was error prone and I found some source code older than compiled code. I introduced eclipse and Ant to reduce human errors. Of course, it took longer to implement the task, but it pays for itself in the long run!
Oh yeah, we did solve the impasse at the warehouse in NJ, but it needed some thinking outside the box. I got the blessing from the local business director and with the help from users and corporate developers, we rolled the users to corporate mainframe (of course, after making necessary changes on the mainframe to accommodate them) and wheeled the AS/400 out, the one I was there to support in the first place! The company saved a bundle and everyone was happy!
Well, there is a lot to talk about that software and the improvements we did. I will do so in other posts. In this post, I want to focus on the issue I see in every software maintenance projects - the status quo in keeping the software architecture intact, irrespective how much it changes in business functionality.
I frequently compare software to a car. The comparison arises in that both are built, run, need to be maintained, and tuned up. That's where the comparison ends. We don't keep adding functionalities to a car. We add bells and whistles like tint, color, alarm etc, but never add functionality or change it's function completely; we don't make a car run on square wheels or sideways. Though the engine gets hot, we don't use a it as a toaster or to cook chicken. But, in software we do exactly that.
When a custom software is designed or purchased, the users typically get all the functions they need. But, over the years users change and so does the business. We keep adding enhancements, bug fixes to the software constantly to meet the (sometimes unreasonable) business requirements that were never dreamed of, when we built the original software. The software thus gets bloated over a time period. We don't adjust the technical architecture or design to match this new software obesity and it eventually comes to a crawl, at which point everyone blames the software and want to replace it. By then, it may be too late! Many custom software out there have nice front ends, but - pardon my French - crappy code inside. It's like a flash new bus pulled by a mule, no offence to the mule!! I blame this on Software Status Quo! We tend to cater to business so much, we neglect the software itself in the process!!
Lot of this is due to the way companies budget for Software projects. Several departments tend to have a "lights-on" policy, which means keeping the software up and running 24X7 without any outage. Companies typically have a lot of budget for "operations" but not real software maintenance. As a consultant, I've seen IT departments struggle to define my work as part of "lights on", so they could continue to get funds for my contract.
Typically, the business side budget pays for the "operations". Except for a handful of savvy business managers, rest of them don't really understand software. To them, the "pretty" screens and "nice" reports are the software. As long as these happen, software is working fine. When you work with such managers, it is not easy to get budget for retrofitting software. "Business" requirements will always take precedence to technical requirements.
But, business is not entirely to blame for this. Recently, when I started introducing the changes in our software, I saw advantages of using more Java in a mostly Powerbuilder environment. There was some awe initially, then reluctance and finally an indifferent acceptance. New architecture means, new problems and tougher maintenance, at least initially. Now, I see why this status quo is continuing. I always hear senior programmers say, if it's not broken and/or users didn't want it, we wouldn't fix it. Where as, this is a safe approach to keep business happy, it hurts the software in the long run.
To end such status quo, business must stop treating software like stationary and recognize and allocate budget for it to be "tuned" regularly. I also feel, real end users (not just business managers) must be regularly consulted as to the performance and usage of the application. I've seen many a software projects fail, in spite of extensive design and funding, because end users simply rejected the software. Developers must also constantly look for ways to change and improve the software itself (not just it's business functionality) and strive to include these in their releases. I see several developers less motivated with the routine business fixes. Adding in technical items, may help to re-energize the developer base.
Many years ago, I worked in a warehouse in NJ where they were using an AS/400 for order tracking, but the software itself was so outdated, the users were improvising. Believe it or not, they would print a report out of this system, then they would input that information into System/36 along with accounting information and export to a file. The next smart user takes that to mainframe to finish up the final entry, where corporate wanted the original shipping and accounts information. Those systems people in Corporate, 3000 miles away conveniently ignored this status quo, that this process actually continued for many years until truly yours stepped in! Ironically, around the same time, corporate headquarters poured in millions to port some of their mainframe applications to flashy client/server tools.
I normally keep notes of all technical issues or a wish list for the software I work on. Then when I get to work on some business requirement in that software, I include any appropriate technical fix. Recently, while working on a business requirement involving Java stored procedures, I rewrote the logging mechanism completely, so we can support it better. Previously, developers were compiling and distributing this code to DBAs completely outside of the build process. If one class had to be updated, just compile it and patch it into the Jar file!!! This was error prone and I found some source code older than compiled code. I introduced eclipse and Ant to reduce human errors. Of course, it took longer to implement the task, but it pays for itself in the long run!
Oh yeah, we did solve the impasse at the warehouse in NJ, but it needed some thinking outside the box. I got the blessing from the local business director and with the help from users and corporate developers, we rolled the users to corporate mainframe (of course, after making necessary changes on the mainframe to accommodate them) and wheeled the AS/400 out, the one I was there to support in the first place! The company saved a bundle and everyone was happy!
Sunday, January 6, 2013
Ubuntu: File Managers
I am continuing to use Ubuntu as my primary desktop and I am loving it(I still use Windows desktop. It has it's advantages). As I get used to the environment, I am always looking to mimic familiar options from Windows. This time around I was looking to find out about adding applications to Context menu in Ubuntu. Like always, I stumbled on something bigger - File Manager(s).
Earlier I posted about Linux architecture and how different layers of the Operating System can be replaced/added. If you read that post, you will see how I struggled with Unity Desktop that came with Ubuntu 12.04 and eventually learned to replace it with Gnome desktop. Apparently, each desktop has it's own flavor of File Manager and then some.
If you are coming from the Windows world (like I did), here are some terminologies for you. What we call Windows in Windows Operating System (like Windows XP) is really the Desktop. We saw earlier there are many different Desktops including Gnome, KDE, Unity etc. The Desktops have several parts and one of them is a File Manager. This is called Windows Explorer (it used to be called File Manager in versions before Windows 95). There are choices for Explorer, but I've not seen many people replacing the Windows Explorer with another.
In Ubuntu (for that matter entire *nix [Unix, Linux]) world, everything is customizable and/or replaceable. I was able to replace my troubled Unity desktop with Gnome desktop and overcome some issues earlier. When I started researching on Context Menus, I stumbled on Nautilus package, which is the default File Manager in Gnome. Since Ubuntu 12.04 didn't support Gnome desktop natively, it didn't install Nautilus either. It had a File Manager called Thunar instead. And, when I installed Gnome, unknowingly, I had also opted for Nautilus, default file manager for Gnome. Apparently, there are at least 20 different File Managers available. See this blog.
This site has a nice picture of the Linux Architecture.
In Gnome Desktop, you get to File Manager by clicking on Places menu on top.
Fig 1: Menu to reach File Manager
Fig 2: Nautilus File Manager
So what about Context Menu?
Coming back to adding context menu, it's done using Nautilus Actions.I had to install it first. (This you could do it in Ubuntu Software Center or use apt-get command). Once you installed it, you can add context menu items as shown below:
Fig 3: Nautilus Actions Configuration
And of course, after changing the context menu you need to reload the Nautilus File Manager. To do this, you can type the below command:
<code>killall nautilus</code>
This kills all processes associated with the particular program (here nautilus). For Windows users, this is like going to Windows Task Manager and doing "End Program". See here for more on killall. Incidentally, they also talk about killing nautilus!!
Running KDiff3 from Context Menu
The program I was trying to add to context menu was KDiff3 (it's a great diff utility, try it, if you haven't already. I use it on Windows as well). Nautilus Actions allows you to add programs to the context menu. I select 2 files to be compared and right click to select and open KDiff3 and voilà! KDiff3 opens and diff's the files automatically!
References
Earlier I posted about Linux architecture and how different layers of the Operating System can be replaced/added. If you read that post, you will see how I struggled with Unity Desktop that came with Ubuntu 12.04 and eventually learned to replace it with Gnome desktop. Apparently, each desktop has it's own flavor of File Manager and then some.
If you are coming from the Windows world (like I did), here are some terminologies for you. What we call Windows in Windows Operating System (like Windows XP) is really the Desktop. We saw earlier there are many different Desktops including Gnome, KDE, Unity etc. The Desktops have several parts and one of them is a File Manager. This is called Windows Explorer (it used to be called File Manager in versions before Windows 95). There are choices for Explorer, but I've not seen many people replacing the Windows Explorer with another.
In Ubuntu (for that matter entire *nix [Unix, Linux]) world, everything is customizable and/or replaceable. I was able to replace my troubled Unity desktop with Gnome desktop and overcome some issues earlier. When I started researching on Context Menus, I stumbled on Nautilus package, which is the default File Manager in Gnome. Since Ubuntu 12.04 didn't support Gnome desktop natively, it didn't install Nautilus either. It had a File Manager called Thunar instead. And, when I installed Gnome, unknowingly, I had also opted for Nautilus, default file manager for Gnome. Apparently, there are at least 20 different File Managers available. See this blog.
This site has a nice picture of the Linux Architecture.
In Gnome Desktop, you get to File Manager by clicking on Places menu on top.
Fig 1: Menu to reach File Manager
Fig 2: Nautilus File ManagerSo what about Context Menu?
Coming back to adding context menu, it's done using Nautilus Actions.I had to install it first. (This you could do it in Ubuntu Software Center or use apt-get command). Once you installed it, you can add context menu items as shown below:
Fig 3: Nautilus Actions ConfigurationAnd of course, after changing the context menu you need to reload the Nautilus File Manager. To do this, you can type the below command:
<code>killall nautilus</code>
This kills all processes associated with the particular program (here nautilus). For Windows users, this is like going to Windows Task Manager and doing "End Program". See here for more on killall. Incidentally, they also talk about killing nautilus!!
Running KDiff3 from Context Menu
The program I was trying to add to context menu was KDiff3 (it's a great diff utility, try it, if you haven't already. I use it on Windows as well). Nautilus Actions allows you to add programs to the context menu. I select 2 files to be compared and right click to select and open KDiff3 and voilà! KDiff3 opens and diff's the files automatically!
References
- http://www.tuxarena.com/2011/06/20-file-managers-for-ubuntu/
- http://www.techdrivein.com/2010/05/what-is-nautilus-elementary-and-how-to.html
- https://live.gnome.org/Nautilus/Screenshots
- http://www.tellmeaboutlinux.com/content/linux-architecture
- http://en.wikipedia.org/wiki/File_manager
- http://askubuntu.com/questions/88480/adding-extra-options-to-right-click-menu
- http://www.omgubuntu.co.uk/2011/12/how-to-add-actions-emblem-support-back-to-nautilus-in-ubuntu-11-10/
- http://www.linfo.org/killall.html
Monday, December 24, 2012
Merry Christmas!
May the Christmas Cheer spread around the world and bring peace, health and wealth to every
one!I just found this nice site to create fancy fonts!
Thursday, November 22, 2012
Now you know: Serifs and Fonts
Have you ever felt like you always knew something and turns out you didn't? Paul Harvey used to host a show called "Rest of the story" on the radio. He would take a familiar subject and give you a little known fact or story about the subject. If Paul was alive and decided to run such a show on technology, he would never run out of ideas for the show. There are several terminologies we take for granted on a daily basis. Some of them are real simple and others more complex. I will share my list of such stories here. If you know of any, please feel free to add them as comments below.
I've always taken HTML and CSS for granted. So, I decided to learn it in a proper way. Who better to learn than w3schools (W3 stands for WWW)? While I was going through their chapters, ended up on a topic about Fonts, that caught my attention and this is the basis of my post today.
I use various fonts, like Arial, Times New Roman, MS-Sans-Serif etc. Didn't pay much attention to the names or how they looked. These Fonts vary in size and shape. Some of them are basic fonts and there are fancy fonts. And of course, there are various other characteristics of a font that make them look different from each other. One such Characteristic is a "Serif". According to Merriam-Webster Dictionary, Serif is actually a word (noun) that means the little edgers added to letters when we write them. Where as, San-Serif would mean the opposite, those fonts without Serif's!! (I don't know if they teach this in English classes in schools, but this is the first time I came across this).
References:
http://www.w3schools.com/css/css_font.asp
http://en.wikipedia.org/wiki/Serif
http://www.merriam-webster.com/dictionary/serif
http://www.merriam-webster.com/dictionary/sans
I've always taken HTML and CSS for granted. So, I decided to learn it in a proper way. Who better to learn than w3schools (W3 stands for WWW)? While I was going through their chapters, ended up on a topic about Fonts, that caught my attention and this is the basis of my post today.
I use various fonts, like Arial, Times New Roman, MS-Sans-Serif etc. Didn't pay much attention to the names or how they looked. These Fonts vary in size and shape. Some of them are basic fonts and there are fancy fonts. And of course, there are various other characteristics of a font that make them look different from each other. One such Characteristic is a "Serif". According to Merriam-Webster Dictionary, Serif is actually a word (noun) that means the little edgers added to letters when we write them. Where as, San-Serif would mean the opposite, those fonts without Serif's!! (I don't know if they teach this in English classes in schools, but this is the first time I came across this).
So next time, you pick a font, enlarge it and see if it has "serifs". As shown Times New Roman font does and Arial doesn't.
Apart from these two types, we also have Monospace fonts that have constant width for all characters. Courier New is of this type and is typically used by text editor and program (code) editors. "Courier New" uses Serifs to adjust the widths to be uniform. Check it out.
Now you know (the rest of the story)!
References:
http://www.w3schools.com/css/css_font.asp
http://en.wikipedia.org/wiki/Serif
http://www.merriam-webster.com/dictionary/serif
http://www.merriam-webster.com/dictionary/sans
Thursday, November 1, 2012
Quicktip - MS Word: How to Paste/format code?
Have you ever tried to paste code into a word document? Ugly! It loses formatting, wraps code around, loses syntax coloring etc. Today, I tried to document my sample Java/JSP code in word. I wanted to intersperse code with additional comments explaining the code for other developers, the way some web pages do it. I tried pasting code into word, it lost formatting and also syntax coloring. I searched good old Google and landed on this page in Stackoverflow (SOF). Out of the several tips, the one using NotePad++ (Npp) caught my attention.
Export to HTML using Npp
I use Npp both at work and home and it's a great tool. Why not use it? I had an older version. Downloaded and installed version 6.2. They have added few more plugins and options. Now, I can colorize JSP code (though I don't like their choice of colors for HTML elements). Per tip in SOF, I tried Npp Export plugin to copy RTF to Clipboard option. This didn't work OK (colors ran in word). I tried the other options. Of all, the "Export to HTML" option worked out the best for me. Colors softened in HTML. Then I just opened the HTML file in Word, saved as word and got down to business with documenting the code. To add flavor, I added a Style called "Code" in Word to customize further. I added "box" to this style and the results are shown below.
Adding Line Numbers
NppExport does not add line #s by default (There is a "feature request" opened for this issue!). I poked around the web for line #s in Npp and found this tip. Essentially, use TextFx Tools to Insert line #s into the file in Npp. Then do the Export like above. Bingo! I've got formatted/syntax highlighted code with line #s pasted into word document. And one more kudos to Npp for job well done!
Fig 1 - Sample code snippet formatted in Word with description around it
Export to HTML using Npp
I use Npp both at work and home and it's a great tool. Why not use it? I had an older version. Downloaded and installed version 6.2. They have added few more plugins and options. Now, I can colorize JSP code (though I don't like their choice of colors for HTML elements). Per tip in SOF, I tried Npp Export plugin to copy RTF to Clipboard option. This didn't work OK (colors ran in word). I tried the other options. Of all, the "Export to HTML" option worked out the best for me. Colors softened in HTML. Then I just opened the HTML file in Word, saved as word and got down to business with documenting the code. To add flavor, I added a Style called "Code" in Word to customize further. I added "box" to this style and the results are shown below.
Adding Line Numbers
NppExport does not add line #s by default (There is a "feature request" opened for this issue!). I poked around the web for line #s in Npp and found this tip. Essentially, use TextFx Tools to Insert line #s into the file in Npp. Then do the Export like above. Bingo! I've got formatted/syntax highlighted code with line #s pasted into word document. And one more kudos to Npp for job well done!
Fig 1 - Sample code snippet formatted in Word with description around it
Friday, December 9, 2011
Visual Expert survey on Powerbuilder
If you are interested in knowing where PB stands out there, there is a survey on PB here:
http://www.visual-expert.com/EN/survey-pb-stored-procedure-sybase-mssql/survey-powerbuilder-2011-us.php
This is the company that makes Visual Expert for PB, we saw earlier. If you take the survey, they let you see their survey results for previous years.
http://www.visual-expert.com/EN/survey-pb-stored-procedure-sybase-mssql/survey-powerbuilder-2011-us.php
This is the company that makes Visual Expert for PB, we saw earlier. If you take the survey, they let you see their survey results for previous years.
Wednesday, June 22, 2011
Google Sites
At work we switched to Google mail some time back. We use Google docs to share docs. I started using Google sites just out of curiosity. Anyone that has a GMail account has automatic access to Google Docs and Google Sites. Google Sites is a simple web site builder. Google Site started as JotBot, a wiki tool that Google bought in 2006 and later developed into what it is today. See wiki article at http://en.wikipedia.org/wiki/Google_Sites for more info.
Creating a Site
Building a site using Google Site is very easy. Simply click on "Create New Site" button and select any site template available in the template gallery. Then select the Theme and visibility settings, voila... you have the new site up and running. Though, it can be used to build any web site, because of it's original wiki flavor, it's a great tool to create wiki pages. Google site also offers several gadgets that can be embedded thus you can include other web pages, graphics etc. Google Docs can easily be embedded as well.
Creating Web Pages
Once a site is created, pages can be added to it at any time. Google Site by default offers the following page templates:
Web Page
Announcements
File Cabinets
etc
Any of these pages can have embedded content. "Web page" template A simple web page. Each new posting to the site will be an additional page. "Announcements" template offers a Blog/wiki style post where each page will have multiple posts. So a wiki site may contain one page of type "Announcements" and multiple posts in it. It's like a blog, except the pages can be added/updated by anyone (See security settings) and thus it's like a wiki page. File Cabinets shows list of files uploaded. One can create other complex templates based on these.
Each page can have comments and attachments. (These options can be turned off for specific pages as well).
Managing the Site
Site settings can be changed at any time, by clicking on More Actions->Mange Site (Only owners can manage sites). This screen can be reached by clicking on "Edit Sidebar" link. Here several options are shown. Here several options are available. In Site Layout option, you can change the size and layout of the site. This also has a Navigation section. By adding pages (links) here, they can be displayed on the side bar. The theme used for the site can be changed at any time as well.
Security Settings
Google Site access is controlled by "Site Permissions". This can changed at any time. Typically permissions are,
1. Public - Any one can see it
2. Anyone with the link - If someone got hold of the link, they can see it
3. Private - Access only by invite; sign-in required to see the pages.
Google Site vs Wordpress
In many ways, Google Sites can be compared to Wordpress.com blog pages. In both, you can create (web) pages quickly and use gadgets ( extensions) to add functionality. Both provide templates and themes to create web pages quickly. In general, Wordpress has a bigger developer community and is also a more matured product. It also has a lot more extensions available. Sites developed using Google Sites tend to be small personal sites. Since Google Sites had a wiki beginning it is stronger on collaboration of content. It allows each site to have multiple owners, contributors and viewers. This makes it more useful for development community to share information within a team.
My Google Sites
I created my first Google Site for our apartment community. It's a small community and I just created a page or 2 as a bulletin board. After that, I created and maintain a web page for our team to use at work. We create some developer documents which we used to dump on Network drive. The problem there was the files are not easily searchable and thus, after 100's files in the drive, we had to rely on individual's memory and notes. I tried to convert these to posts in the Google Site.
The current site I maintain at work has several pages, some with embedded Google Docs (converted Word, Powerpoint, Excel files). I also used a embedded widget to included an IFrame, so I can display an external web site. Since it's meant to be a collaborative site, I created several pages for various topics, each using "Announcement" template, so several posts can be made to the same page. For e.g., I have PB Wiki, for all posts related to our Powerbuilder development environment. Similarly a "Database Wiki" page for database related postings.
Now that we have site up and running, we need to back it up. I will discuss it in a separate post.
Creating a Site
Building a site using Google Site is very easy. Simply click on "Create New Site" button and select any site template available in the template gallery. Then select the Theme and visibility settings, voila... you have the new site up and running. Though, it can be used to build any web site, because of it's original wiki flavor, it's a great tool to create wiki pages. Google site also offers several gadgets that can be embedded thus you can include other web pages, graphics etc. Google Docs can easily be embedded as well.
Creating Web Pages
Once a site is created, pages can be added to it at any time. Google Site by default offers the following page templates:
Web Page
Announcements
File Cabinets
etc
Any of these pages can have embedded content. "Web page" template A simple web page. Each new posting to the site will be an additional page. "Announcements" template offers a Blog/wiki style post where each page will have multiple posts. So a wiki site may contain one page of type "Announcements" and multiple posts in it. It's like a blog, except the pages can be added/updated by anyone (See security settings) and thus it's like a wiki page. File Cabinets shows list of files uploaded. One can create other complex templates based on these.
Each page can have comments and attachments. (These options can be turned off for specific pages as well).
Managing the Site
Site settings can be changed at any time, by clicking on More Actions->Mange Site (Only owners can manage sites). This screen can be reached by clicking on "Edit Sidebar" link. Here several options are shown. Here several options are available. In Site Layout option, you can change the size and layout of the site. This also has a Navigation section. By adding pages (links) here, they can be displayed on the side bar. The theme used for the site can be changed at any time as well.
Security Settings
Google Site access is controlled by "Site Permissions". This can changed at any time. Typically permissions are,
1. Public - Any one can see it
2. Anyone with the link - If someone got hold of the link, they can see it
3. Private - Access only by invite; sign-in required to see the pages.
Google Site vs Wordpress
In many ways, Google Sites can be compared to Wordpress.com blog pages. In both, you can create (web) pages quickly and use gadgets ( extensions) to add functionality. Both provide templates and themes to create web pages quickly. In general, Wordpress has a bigger developer community and is also a more matured product. It also has a lot more extensions available. Sites developed using Google Sites tend to be small personal sites. Since Google Sites had a wiki beginning it is stronger on collaboration of content. It allows each site to have multiple owners, contributors and viewers. This makes it more useful for development community to share information within a team.
My Google Sites
I created my first Google Site for our apartment community. It's a small community and I just created a page or 2 as a bulletin board. After that, I created and maintain a web page for our team to use at work. We create some developer documents which we used to dump on Network drive. The problem there was the files are not easily searchable and thus, after 100's files in the drive, we had to rely on individual's memory and notes. I tried to convert these to posts in the Google Site.
The current site I maintain at work has several pages, some with embedded Google Docs (converted Word, Powerpoint, Excel files). I also used a embedded widget to included an IFrame, so I can display an external web site. Since it's meant to be a collaborative site, I created several pages for various topics, each using "Announcement" template, so several posts can be made to the same page. For e.g., I have PB Wiki, for all posts related to our Powerbuilder development environment. Similarly a "Database Wiki" page for database related postings.
Now that we have site up and running, we need to back it up. I will discuss it in a separate post.
Monday, June 20, 2011
Pinging windows PC behind firewall
If your windows PC (XP, Vista) has firewall turned on, it may not respond to ping from other computers on the network. To allow pinging on this computer, try the following command:
To disable the same,
Make sure you open command prompt as an Administrator.
Netsh is the command line utility available in windows command prompt (DOS box) to help with network configuration. See here for more on netsh.
The above settings can also be set in windows firewall settings GUI (screen) in control panel. Surprisingly it's a little more convoluted. See here
netsh firewall set icmpsetting 8To disable the same,
netsh firewall set icmpsetting 8 disableMake sure you open command prompt as an Administrator.
Netsh is the command line utility available in windows command prompt (DOS box) to help with network configuration. See here for more on netsh.
The above settings can also be set in windows firewall settings GUI (screen) in control panel. Surprisingly it's a little more convoluted. See here
Dot... and Dotty
As a developer, one of the things I constantly do is documentation. This is not exactly a favorite task of mine, but I know it's a necessary evil. My programs are typically commented generously. Where necessary I create separate technical documents as well.
In order to make this mundane task a little more interesting, I always try to find tools to help with documentation. For e.g., I found Cppdoc, to document C++ programs, (similar to Javadoc), couple of years ago. While researching on this, I stumbled on Dotty. I found a great tool called GraphViz, but I didn't do much with it at the time.
Recently when I had a need to create a flowchart for a work flow, again I looked for some tools. I could have used Visio and created some diagrams, but I wanted to automate the process of creating diagrams based on some (text) input. While researching, I re-discovered Dot and Dotty!
DOT and Dotty were originally developed by AT&T Bell Labs. DOT is a plain text graph description language. It's a definition language like CSS. We can simply type up the information about a graph in human readable Text format, then feed it to a tool to generate a visual graph in various graphic formats. Dotty is a graph editor. AT&T actually bundled dotty and other tools as Graphviz. Graphviz is a collection of tools that interpret Dot structures and generate Graphs in various layouts.
In Dot language, Graphs can be defined as directed (DiGraph) or undirected (Graph). It also defines various elements in a graph such as nodes, arrows/lines or edges, labels etc. The syntax for the language includes various attributes of these elements as well. Thus by combining these elements and their attributes, one can build complex graphs/charts. Examples of such diagrams include, database schema diagrams, network nodes and connections, flow charts etc. The tools are capable of generating the diagrams in various graphic formats including JPG, PNG and SVG. A diagram is created in SVG format can be read by visio and other tools, thus sharing is easier.
The specification for DOT language is simple:
For e.g.,
Each circle there is a node and the lines (with or without arrows) are called edges. Each of these have several attributes which can be used to control the presentation and to some extent actual layout.
Below is a simple directional graph (lines with arrows):
digraph{A->B->C->A}
Dotty and other tools in Graphviz bundle all have similar command line syntax. A typical usage would be,
dot -O -Tpng <example.dot>
I found a great use for this tool, recently. At work, we have sort of a workflow system. The system keeps track of various types of Cases generated by the system or created by the users. Each case type is defined by it's own workflow. To track various steps in the workflow, we used a Treeview structure in Powerbuilder.While I was doing some research on these flows, I had a need to document the flows in a flowchart. As always, I didn't want to just open Visio and draw a few. I wanted to script it, so it can be generated by text and better yet, automatically from the database with the information we have about the flows!
Since it's so easy to create graphs from the text files, we may even be able to build graphs dynamically, thus could represent state diagrams of running processes.
There are various resources available for DOT and dotty. Dotty is originally written in C. There is a Java API (Grappa) available that wraps around Dotty functions. Apart from Dot, there are also other languages available for representing graphs in text format.
References:
http://www.graphviz.org/Documentation.php
http://wapedia.mobi/en/DOT_language
http://code.google.com/apis/chart/image/docs/gallery/graphviz.html
In order to make this mundane task a little more interesting, I always try to find tools to help with documentation. For e.g., I found Cppdoc, to document C++ programs, (similar to Javadoc), couple of years ago. While researching on this, I stumbled on Dotty. I found a great tool called GraphViz, but I didn't do much with it at the time.
Recently when I had a need to create a flowchart for a work flow, again I looked for some tools. I could have used Visio and created some diagrams, but I wanted to automate the process of creating diagrams based on some (text) input. While researching, I re-discovered Dot and Dotty!
DOT and Dotty were originally developed by AT&T Bell Labs. DOT is a plain text graph description language. It's a definition language like CSS. We can simply type up the information about a graph in human readable Text format, then feed it to a tool to generate a visual graph in various graphic formats. Dotty is a graph editor. AT&T actually bundled dotty and other tools as Graphviz. Graphviz is a collection of tools that interpret Dot structures and generate Graphs in various layouts.
In Dot language, Graphs can be defined as directed (DiGraph) or undirected (Graph). It also defines various elements in a graph such as nodes, arrows/lines or edges, labels etc. The syntax for the language includes various attributes of these elements as well. Thus by combining these elements and their attributes, one can build complex graphs/charts. Examples of such diagrams include, database schema diagrams, network nodes and connections, flow charts etc. The tools are capable of generating the diagrams in various graphic formats including JPG, PNG and SVG. A diagram is created in SVG format can be read by visio and other tools, thus sharing is easier.
The specification for DOT language is simple:
For e.g.,
graph {
a--b--c
}
Will generate
Each circle there is a node and the lines (with or without arrows) are called edges. Each of these have several attributes which can be used to control the presentation and to some extent actual layout.
Below is a simple directional graph (lines with arrows):
digraph{A->B->C->A}
Following line creates a yellow colored oval node:
n[style=filled color="red" fillcolor="yellow"]
The same node as a rectangle:
n[shape=rectangle style=filled color="red" fillcolor="yellow"]
There are several other attributes that can be used to build really complex diagrams. See here for syntax. References listed at the bottom have more details.
The strength of Dot is that the language is small and text-based. Thus, we can write scripts or programs to generate such a text description of a graph. One such application would be to generate graphical calling tree for programs.
Dotty and other tools in Graphviz bundle all have similar command line syntax. A typical usage would be,
dot -O -Tpng <example.dot>
I found a great use for this tool, recently. At work, we have sort of a workflow system. The system keeps track of various types of Cases generated by the system or created by the users. Each case type is defined by it's own workflow. To track various steps in the workflow, we used a Treeview structure in Powerbuilder.While I was doing some research on these flows, I had a need to document the flows in a flowchart. As always, I didn't want to just open Visio and draw a few. I wanted to script it, so it can be generated by text and better yet, automatically from the database with the information we have about the flows!
Since it's so easy to create graphs from the text files, we may even be able to build graphs dynamically, thus could represent state diagrams of running processes.
There are various resources available for DOT and dotty. Dotty is originally written in C. There is a Java API (Grappa) available that wraps around Dotty functions. Apart from Dot, there are also other languages available for representing graphs in text format.
References:
http://www.graphviz.org/Documentation.php
http://wapedia.mobi/en/DOT_language
http://code.google.com/apis/chart/image/docs/gallery/graphviz.html
Friday, March 11, 2011
PVCS - Source repository File Server
In PVCS, when we try to open the project database, it typically lists File Servers to select. The file server name listed there is not the actual server's name. Recently, we had to find out what server it was connecting to. This can be done in Admin->File Servers menu option in PVCS. You can add/delete/update server names here.
[caption id="attachment_4952" align="alignnone" width="300"]
Where to add PVCS server list[/caption]
So, if the PVCS server is changed in the future and you need to connect to the new server, change it as mentioned above. This needs to be done on each developer machine or the ini file could be distributed. (The name of the ini file is, C:\Program Files\Merant\vm\common\bin\win32\servers.ini)
[caption id="attachment_4951" align="alignnone" width="300"]
Fig 2. server list is stored in servers.ini[/caption]
If you connect to PVCS from Powerbuilder (or any other MSSCCI compliant IDEs), you do not have to change anything there. PB actually reads the PVCS servers.ini file to get the PVCS Server name(s). After making the changes to File Server name in PVCS, we just need to disconnect and reconnect to merant version manager in Powerbuilder.
[caption id="attachment_4952" align="alignnone" width="300"]
Where to add PVCS server list[/caption]So, if the PVCS server is changed in the future and you need to connect to the new server, change it as mentioned above. This needs to be done on each developer machine or the ini file could be distributed. (The name of the ini file is, C:\Program Files\Merant\vm\common\bin\win32\servers.ini)
[caption id="attachment_4951" align="alignnone" width="300"]
Fig 2. server list is stored in servers.ini[/caption]If you connect to PVCS from Powerbuilder (or any other MSSCCI compliant IDEs), you do not have to change anything there. PB actually reads the PVCS servers.ini file to get the PVCS Server name(s). After making the changes to File Server name in PVCS, we just need to disconnect and reconnect to merant version manager in Powerbuilder.
Thursday, March 10, 2011
TomCat + Sybase EA Server
Today another coworker of mine was trying to get XAMPP setup work with our EA Server. (We have PB application and Java objects deployed to this EA Server). He has a complete setup of XAMPP with Apache and Tomcat. He has a JSP page that uses the PB objects. This got to a point and crashed with "class not found" error. Clearly pointing to a classpath issue. But which one?
He mentioned that he had to set the EA Server first before he started XAMPP. Clearly there are some conflicts in the classes shared between the 2. EA Server is normally started using a batch file that reads the env classpath (windows settings in control panel) and adds its own classpaths locally. (SET LOCAL inside batch file). So whatever EA Server set is not visible to XAMPP but not the other way around. That's why he had to start EA Server first.
We then looked for the specific class that was "not found". This happened to a stub for the PB objects. In Jaguar manager (aka Sybase Central, EA Server admin) there is an option to generate stubs (and skeletons if needed). This typically points to SYBASEEASERVERHTMLCLASSES directory. We found out that he also similar classes in SYBASEEASERVERJAVACLASSES directory. We knew these were not used, as they were not in the class path. Then we searched for classes or jar files with similar names. We stumbled on a jar file that contains part of these stubs, inside XAMPP. (He uses OpenBD for coldfusion applications, which is why he has XAMPP in the first place). This jar was old and had part of the stubs classes and did not have the one we were having trouble with.
Apparently Tomcat was automatically loading this jar (Anything inside the lib directory gets loaded automatically). We found that when we tried to rename the file.
Now that we found out the classes in trouble, we thought we could just add the class paths needed to the environment classpath and hope Tomcat would pick it up. This did not happen. On the web, I saw a note about how Tomcat handles classpaths differently than regular Java. There was a suggestion to the classpath to the batch file that starts Tomcat. We didn't try that yet.
For now, we decided to copy the classes to the library folder, so Tomcat can automatically load the classes. So, we shut down XAMPP, regenerated the jar file (in EA Server stub generation, there is an option to generate a jar instead of class files) and copied to XAMPP lib folder and everything worked OK.
He mentioned that he had to set the EA Server first before he started XAMPP. Clearly there are some conflicts in the classes shared between the 2. EA Server is normally started using a batch file that reads the env classpath (windows settings in control panel) and adds its own classpaths locally. (SET LOCAL inside batch file). So whatever EA Server set is not visible to XAMPP but not the other way around. That's why he had to start EA Server first.
We then looked for the specific class that was "not found". This happened to a stub for the PB objects. In Jaguar manager (aka Sybase Central, EA Server admin) there is an option to generate stubs (and skeletons if needed). This typically points to SYBASEEASERVERHTMLCLASSES directory. We found out that he also similar classes in SYBASEEASERVERJAVACLASSES directory. We knew these were not used, as they were not in the class path. Then we searched for classes or jar files with similar names. We stumbled on a jar file that contains part of these stubs, inside XAMPP. (He uses OpenBD for coldfusion applications, which is why he has XAMPP in the first place). This jar was old and had part of the stubs classes and did not have the one we were having trouble with.
Apparently Tomcat was automatically loading this jar (Anything inside the lib directory gets loaded automatically). We found that when we tried to rename the file.
Now that we found out the classes in trouble, we thought we could just add the class paths needed to the environment classpath and hope Tomcat would pick it up. This did not happen. On the web, I saw a note about how Tomcat handles classpaths differently than regular Java. There was a suggestion to the classpath to the batch file that starts Tomcat. We didn't try that yet.
For now, we decided to copy the classes to the library folder, so Tomcat can automatically load the classes. So, we shut down XAMPP, regenerated the jar file (in EA Server stub generation, there is an option to generate a jar instead of class files) and copied to XAMPP lib folder and everything worked OK.
Sunday, February 15, 2009
Fixing an old computer!
My neighbor brought his computer to me about a month ago! First, I thought it was just the video card. So, I bought one through e-bay. Nop. Then I am on to CPU. I refuse to accept it may be the motherboard! This particular model doesn't have a model #. Only a serial #. Compaq/HP online support was so good. He was able to find out the details of the machine. From there I learned more and more about it.
I gathered it was 1.1 GHz celeron processor. So, I set out to work. I ordered, again on ebay, a CPU that looked similar. Least I knew, not all 1.1 GHZ celeron processors were the same. They looked the same (green top and all), but since I had not looked into the CPU before, I assumed all were alike.
The new CPU arrived last week. I finally removed the clip on the heat sink. This was interesting. I hit the mother board a few times with the screwdriver. I hope I didn't damage it. The clip was to be pushed down and twisted to pull it out the plastic tab holding it.
So, I set out to go Fry's electronics yesterday. I bought a few things:
1. Anti static arm band
2. Antistatic gloves
3. thermal grease remover, cleaner
4. thermal paste, anatec silver
This is becoming an expensive affair.
The following websites were really useful in finding out more about celeron cpu's in general and about removing/replacing cpu/heat sink or fan.
Celeron models: There are differences!!!
http://www.hardwaresecrets.com/article/187/6
http://www.intel.com/support/processors/pentiumiii/sb/CS-007568.htm
Removing and applying thermal paste:
http://www.softhardware.net/guides/hardware/raatp.php
And there is even names for these Pentium 3/Celeron sockets:
coppermine (PGA ) and Tualatin (PGA2)
There is even a socket to put the new chip in the old socket!!
http://www.pcpartscollection.com/fctuadforso3.html
http://www.redflagdeals.com/forums/archive/index.php/t-549619.html
http://www.intel.com/support/processors/pentium4/sb/cs-017370.htm
This video was helpful too:
http://www.youtube.com/watch?v=4OB1hxxKdec
I gathered it was 1.1 GHz celeron processor. So, I set out to work. I ordered, again on ebay, a CPU that looked similar. Least I knew, not all 1.1 GHZ celeron processors were the same. They looked the same (green top and all), but since I had not looked into the CPU before, I assumed all were alike.
The new CPU arrived last week. I finally removed the clip on the heat sink. This was interesting. I hit the mother board a few times with the screwdriver. I hope I didn't damage it. The clip was to be pushed down and twisted to pull it out the plastic tab holding it.
So, I set out to go Fry's electronics yesterday. I bought a few things:
1. Anti static arm band
2. Antistatic gloves
3. thermal grease remover, cleaner
4. thermal paste, anatec silver
This is becoming an expensive affair.
The following websites were really useful in finding out more about celeron cpu's in general and about removing/replacing cpu/heat sink or fan.
Celeron models: There are differences!!!
http://www.hardwaresecrets.com/article/187/6
http://www.intel.com/support/processors/pentiumiii/sb/CS-007568.htm
Removing and applying thermal paste:
http://www.softhardware.net/guides/hardware/raatp.php
And there is even names for these Pentium 3/Celeron sockets:
coppermine (PGA ) and Tualatin (PGA2)
There is even a socket to put the new chip in the old socket!!
http://www.pcpartscollection.com/fctuadforso3.html
http://www.redflagdeals.com/forums/archive/index.php/t-549619.html
http://www.intel.com/support/processors/pentium4/sb/cs-017370.htm
This video was helpful too:
http://www.youtube.com/watch?v=4OB1hxxKdec
Subscribe to:
Posts
(
Atom
)