Coding Thoughtfully

E-Commerce Sites

2011-11-05T02:25:00.000-07:00

A few years ago, creating an E-Commerce site would have been a big deal. Technically, you'd probably have started off with Drupal or other website framework, installed it into a web hosting company site, installed a wide range of modules to get product lists, shipping and other features, found and installed a theme, found someone to develop the connection to a payment gateway (or used one of the very few available ones), and then started the fun of testing and debugging.

This would have been, if you were efficient, a day or two of effort just for the setup, then on top would be theming and testing, and the costs involved, including probably experienced developers.

Now, with providers such as http://www.bigcommerce.com/ and http://www.shopify.com/, this has all gone away and to set up a e-commerce store is, honestly, 5 minutes of work. Even better, these providers integrate with such a wide range of payment gateways, it is unlikely you'd have any need for development.

These E-commerce stores are so easy, non-technical people can use them without involving software developers.

This is another example of what was once a complex toolset being made available for trivial costs ($25 per month). It provides everybody wanting to sell goods or services online with an extremely low barrier to entry. And I won't even get on to Etsy.

I had the opportunity to be involved in depth with the setup of an online store using one of these providers recently. Wistful Kiwi's is a store that sells iconic New Zealand food for delivery overseas (Expats take note!). It was much more enjoyable than delving back into the world of Drupal.(side note, what sort of spelling checker doesn't have the word "Zealand" in it?)

Flex, IE and URLStream ProgressEvent events.

2009-09-21T15:24:00.000-07:00

Well, this was good to fix.

One of the systems I'm working on is a Flex (3) application that for one long running activity needs to update the user on the activity progress. To do this, I use a progress bar, filling from 0 to 100%.

To update the progress, I first send the # of events, then I send a '.' character from the server to the client for each activity performed on the server side. I only need to send about 100 '.'s so it's only a small amount of data.

in Flex, I do this using the Flex URLStream() object, like so:

var loader:URLStream = new URLStream();
var allDataRead:String;

loader.addEventListener(Event.COMPLETE, function (event:Event) :void {
// Do completion stuff.
});

var totalEventsExpected : Number = -1;
loader.addEventListener(ProgressEvent.PROGRESS, function (event:ProgressEvent) :void {
if (loader.bytesAvailable >= 7 && totalEventsExpected == -1) {
allDataRead = loader.readUTFBytes(7);
totalEventsExpected = Number(allDataRead);
}
if (totalEventsExpected > 0) {
popup.progress.setProgress(event.bytesLoaded - 7, totalEventsExpected);
}
});

var request:URLRequest = new URLRequest(MYURL + "&");
request.data = params;
loader.load(request);

The object 'popup' is a window with a progress bar which is updated as data comes in.

This works perfectly under FireFox, but under IE, the progress bar doesn't show any progress. The server side works fine, but the user gets no feedback. I.e. the ProgressEvent does not seem to be fired when data comes in.

The reason for this is, it appears, that IE buffers the stream data until a certain amount of data is read, then it provides progress events for bytes that come in after that. That is to say, because I was sending only a small amount of data, all data was being buffered by IE and only the completion event was being fired.

To fix this, and to provide a progressing progress bar under IE, all I had to do was send a bunch of bytes initially to force IE to stop buffering, and provide the progress events. I found that sending 4000 bytes was enough.

So, if you're in the same situation, just send 4000 bytes initially to get things started.

RIA Technology Comparison

2009-03-03T20:52:00.000-08:00

Recently for work I did a quick comparison of RIA technology for a system we're looking at developing. The results can be seen in this Google Spreadsheet.

We ended up choosing to go with Flex + Perl/Apache and a very thin framework. It's quite a nice system to work with.

Solaris, Java, iReport, JasperReports, Oracle and Locales

2009-02-09T21:51:00.000-08:00

Today at work I spent a lot of time discovering the following fun details about getting JasperReports to display localised text. The reports connect to an Oracle database, and use both Java localisation (via NumberFormat.getInstance() etc) and Oracle localisation ('select to_char(the_date, 'Day Month YY') as date') in creating the reports (using JasperReports v3.1.2).

The reports get the locale from the environment (well the java equivalent of -Duser.language and -Duser.country) as well. The reports run in a Solaris 10 environment.

Note that JasperReport PDF output is based on iText.

I wanted to share this in the hope it helps someone.

Locales on Solaris:
===================

In Solaris, to view terminal output in other languages, you need to have installed the right language packs. Ref to:

http://developers.sun.com/dev/gadc/techtips/sol9_locale_pkgs/index.html

for lists of the packages you need to install for each language. For Solaris 10 - go here http://developers.sun.com/global/products_platforms/solaris/reference/faqs/Locale_config_S10.txt-s10u5-sparc.txt

To check whether or not language packs are install, run the command:

locale -a

This will print out a list of the locales, and the character encodings available for those locales. e.g:

$ locale -a
C
POSIX
ar
ar_EG.UTF-8
en_CA
en_CA.ISO8859-1
en_CA.UTF-8
en_US
en_US.ISO8859-1
en_US.ISO8859-15
en_US.ISO8859-15@euro
es
es_MX
es_MX.ISO8859-1
es_MX.UTF-8
fr
fr_CA
fr_CA.ISO8859-1
fr_CA.UTF-8
iso_8859_1
ja
ja_JP.PCK
ja_JP.UTF-8
ja_JP.eucJP

In this example output, I have Japanese, French (Canadian), Spanish (Mexican), English (US and Canadian), and Arabic (Egyption).

The format is location_territory.encoding, with territory and encoding optional. The territory can be important (e.g. there are differences between French and Canadian French), and the encoding is important for getting the data to view properly.

Assuming you have the relevant locales install, you can then set your language in the following way (in the terminal):

$ export LANG=ja_JP.UTF-8
$ export LC_ALL=ja_JP.UTF-8

Then you can try it out, e.g. using the 'date' command:

$ date
2009Ç¯02·î09Æü (·î) 23»þ30Ê¬33ÉÃ GMT

Assuming you're able to view UTF-16, you'll notice that in this case, I'm obviously not getting the right characters through. This is because my terminal emulator (kterm in this case) has not been set to the right character encoding, so it is not correctly identifying the characters coming through. In fact, my terminal doesn't have a UTF-8 version of Japanese available as a
character encoding under the Japanese list. I need to use ja_JP.eucJP:

$ export LANG=ja_JP.eucJP
$ export LC_ALL=ja_JP.eucJP

I then set my terminal emulator to have the right encoding, and then try the
date command again:

$ date
2009年02月09日 (月) 23時41分35秒 GMT

Much better.

(note that changing my character encoding to UTF-8 for my terminal, and
then changing LANG/LC_ALL back to ja_JP.UTF-8 does work as well).

Arabic:

$ export LANG=ar_EG.UTF-8
$ export LC_ALL=ar_EG.UTF-8
$ date
الأثنين, 9 فبراير, 2009

Note that not all programs will localise correctly just because you have locales for them. E.g. X11 will need the relevant fonts, and programs will need localisation packs for translating text and other sorts of things to the right language. E.g. going 'man date' still shows the information in English, despite my language being in Arabic. On the other hand, if I set my language to Japanese on my local Linux machine, and run ireport, I get Japanese throughout the
ireport UI.

The locale approach above is what works for Linux as well.

For command line Java, the locale is important, as is the terminal character encoding! Though if you redirect to a file, then read the file in a text editor that handles UTF (and has the right fonts to display the language characters) then you can see them properly. All in all the key thing here is that you've got to have the same locale and encoding throughout the whole string
of systems.

Right, so that's command line stuff under Solaris working.

PDF Documents and Locales
=========================

When it comes to PDF documents (i.e. the reports I'm wanting), things get tricky and a bit of a pain. Basically PDF documents can assume only a small list of fonts exist, and it just so happens that (as far as I can tell) that the available fonts don't have Arabic (for example) available.

So, reports need to be written to embed a good font within the PDF document. This is not some global thing either - in jasper reports every text field where it is relevant needs to be explicitly altered to use this new font.

To be more annoying, the right PDF font encoding also needs to be chosen, and this can be different dependant on the locale chosen, though I've found Identity-H works well for all my tests.

Note that some locales (ones based on, or very close to, the English character set) don't necessarily have this problem.

So, to make PDF reports generated via iReport work with varying languages, you need to set a bunch of per-field attributes to ensure the right font/font-encoding is available/used when the PDF is viewed. An example of a field might be:

<font fontName="Arial" size="24" pdfFontName="Arial.ttf" pdfEncoding="Identity-H" isPdfEmbedded="true"/>

So set up each text field in this manner to get the right encoding.

Oracle and Locales
==================

Oracle uses the NLS_LANG environment variable to identify the locale when run from the command line. You must set this to match the similar locale you set for LANG and LC_ALL, and of course your terminal must be reading the encoding right.

Common NLS_LANG values can be found:

http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm#_Toc110410556

With the right encoding and right NLS_LANG, you can run sqlplus and get the date that shows up in the right output format:

$ sqlplus test/test

SQL*Plus: Release 10.2.0.2.0 - Production on الثلاثاء فبراير 10 04:42:37 2009

(note that Oracle ignores the LANG/LC_ALL variables, and uses NLS_LANG
exclusively)

BUT, this won't necessarily work when asking Oracle to convert a date for you:

SQL> select to_Char(sysdate, 'Day "and" Month') from dual;

TO_CHAR(SYSDATE,'DA
-------------------
???????? and ??????

Will come up if your database encoding is set to USASCII7:

SQL> select * from v$nls_parameters where parameter like '%SET%';

PARAMETER
----------------------------------------------------------------
VALUE
----------------------------------------------------------------
NLS_CHARACTERSET
US7ASCII

NLS_NCHAR_CHARACTERSET
AL16UTF16

This is bad, as you need the value of NLS_CHARACTERSET to be something
other than US7ASCII, e.g:

SQL> select * from v$nls_parameters where parameter like '%SET%';

PARAMETER
----------------------------------------------------------------
VALUE
----------------------------------------------------------------
NLS_CHARACTERSET
AL32UTF8

NLS_NCHAR_CHARACTERSET
AL16UTF16

AL32UTF8 is the latest and best to use, but if your DB is in USASCII7,
you can't go to AL32UTF8, you need to create a new database and export/
import data (You may be able to go to another char set which may
work fine - I don't know).

The URS database install must be altered to use AL32UTF8, not USASCII7.

Once you have AL32UTF8, you can ask Oracle to convert dates/times fine:

SQL> select to_char(sysdate, 'Day "and" Month') from dual;

TO_CHAR(SYSDATE,'DAY"AND"MONTH')
-------------------------------------------------------------
الثلاثاء and فبراير

Conclusions
===========

What you need to have to make Oracle, JasperReports (iText) PDF output, and Solaris all play nice with internationalisation (i18n) is:

1/ Install Solaris languages e.g. Arabic (For Egypt), Spanish (Spain) and Japanese.

2/ Test that with the right LANG and LC_ALL a 'date' command prints out the right date format and names with no ? or other nonsense chars (with term. emulator using right char encoding as well).

This step isn't strictly necessary if you're not using the command line at all.

3/ Build/rebuild the database with a character set of AL32UTF8, and test that with a NLS_LANG set to "ARABIC_UNITED ARAB EMIRATES.AR8MSWIN1256" or "JAPANESE_JAPAN.JA16SJIS" the right string is printed with:

select to_char(sysdate, 'Day "and" Month') from dual;

Note that LANG/LC_ALL are not relevant for Oracle clients if NLS_LANG is set.

4/ Create and test your jasper reports with:
a/ Text fields set to use the Arial TTF (e.g. the Microsoft one). Other fonts will work as long as they have the right unicode codepoints defined for your language.
b/ Text fields set to embed truetype font
c/ Text fields set to the PDF encoding of "Identity-H"

Note that Japanese for example needs another font such as ArialUni.ttf (unicode Arial). The PDF encoding of Identity-H seems to work for all of them.

Note that all of this doesn't necessarily mean that a language/country encoding will work - Oracle still needs to support it.

Note that while Arial.ttf is (as far as I can tell) free to distribute, ArialUni.ttf is not. Arail is also not exactly a beautiful font.

OpenOffice + KDE hanging on file open/save dialogs

2009-01-29T16:24:00.000-08:00

Upgraded to KDE 4.2 today, and also installed the kde/gtk integration packages* (to allow me to chose the GTK theme when choosing the KDE one).

Anyway, then I found that the file open/save dialog boxes in OpenOffice 3.0 (possibly earlier versions will have the same problem as well) either didn't show up, or when trying to export my document as a pdf, would hang OpenOffice.

The workaround is to, in OpenOffice go to:

Tools -> Options -> OpenOffice.org -> General

And check the "Use OpenOffice.org dialogs"

http://user.services.openoffice.org/en/forum/viewtopic.php?f=16&t=8705 is the only link on the web where I found any info on this.

That makes OO work again. Note that I do have openoffice-org-gtk (and -kde) installed as well, as I used to use Gnome. I suspect if I uninstall the GTK one it'll work.... nope that doesn't help.

[*] I'm not sure if this integration package makes a difference.

Accessing DSE Routers

2008-09-24T02:52:00.000-07:00

An interesting little tidbit of knowledge.

If you have a DSE (Dick Smith Electronics) branded router you may be able to access hidden functionality such as iptables.

Turn on telnet access on the router and then telnet to the router. Ensure you allow telnet access only from the LAN (not the WAN).

Then, once logged in, type 'sh' - if you're lucky this'll give you shell access to the router. While the router may not have many commands, you can go:

cd /bin

echo *

to see the commands available. If you're really lucky, you'll have iptables available, and with this you can do significantly more than what the web interface is likely to provide.

I used this to great effect to provide IPSec passthrough on a DSE ADSL router that didn't provide the functionality via the UI. Saved over $50 and a trip through the rain.

$ telnet 192.168.1.1

Trying 192.168.1.1...

Connected to 192.168.1.1.

Escape character is '^]'.

BCM96338 ADSL Router

Password:

> sh

BusyBox v1.00 (2006.12.21-05:52+0000) Built-in shell (msh)

Enter 'help' for a list of built-in commands.

# cd /bin

# echo *

adsl adslctl atm atmctl brctl busybox cat cfm chmod date ddnsd df dhcpc dhcpd dmesg dnsprobe dumpmem ebtables echo ethctl false iptables kill ln macaddr mkdir mount msh netctl ping pppd ps pvc2684ctl pvc2684d pwd rm sendarp setmem sh snmp sntp sysinfo tftpd true udhcpd upnp

# iptables --list

Chain INPUT (policy ACCEPT)

target prot opt source destination

ACCEPT icmp -- anywhere anywhere icmp echo-request

ACCEPT 2 -- anywhere anywhere

ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED

DROP all -- anywhere anywhere

Chain FORWARD (policy ACCEPT)

target prot opt source destination

ACCEPT esp -- anywhere 192.168.1.4

ACCEPT tcp -- anywhere 192.168.1.2 tcp dpt:www

Connecting OS X to a Linux CUPS Printer Server

2008-09-23T02:13:00.001-07:00

Recently I've had to deal with setting up an OS X (10.4) machine to use a Linux CUPS printer queue.

While not rocket science this did prove to have a simple trick to it. When setting up the printer, follow these steps to ensure that OS X can find the printer.

Step 1: Open Print & Fax (via System Preferences):

Click the [+] button and use the add printer dialog to add the printer. Choose "Internet Printing Protocol - IPP".

It is important to note at this stage that while you must type in a valid IP address (or hostname), it is not actually necessary or used for the CUPS IP printer, the relevant fields are the Name and Location fields.

The Name field gives the printer a nice name (which you'll see in print dialog boxes)
The Location field gives the full location of the printer and (as far as I can tell) is the only field that is relevant for OS X for finding the CUPS printer.

To find out the locations of your printer(s), go to the printer server with your web browser, browsing to port 631. (e.g. the base URL may be http://192.168.1.2:631, and printers will be listed under http://192.168.1.2:631/printers):

The location of a printer is, essentially, the URL that the printer title on this page links to. For example, in the above screenshot "Brother_HL-2040_series_USB_1" is a link, and it is that link that constitutes the printer location.

So, put in the Location field of the add printer dialog the printer URL, without the "http://" bit. For example:

If you can find your printer in the list, then try that. Otherwise using the "Generic PostScript Printer" option seems to work fine.

Add the printer now and you should find you printer now works from OS X.

If you are still unable to print, try and turn in CUPS debug. On the Linux machine, edit the CUPS configuration file (under Ubuntu this is /etc/cups/cupsd.conf) and change the log level to debug (e.g.):

# Log general information in error_log - change "info" to "debug" for

# troubleshooting...

LogLevel info

Would go to:

# Log general information in error_log - change "info" to "debug" for

# troubleshooting...

LogLevel debug

Then restart CUPS:

sudo /etc/init.d/cupsys restart

Then view the CUPS log:

tail -f /var/log/cups/error_log

If you see an error that looks a bit like this (in particular the "client-error-not-found" bit):

D [23/Sep/2008:19:50:47 +1200] cupsdAcceptClient: 8 from 192.168.1.4:631 (IPv4)

D [23/Sep/2008:19:50:47 +1200] cupsdReadClient: 8 POST / HTTP/1.1

D [23/Sep/2008:19:50:47 +1200] cupsdAuthorize: No authentication data provided.

D [23/Sep/2008:19:50:47 +1200] Get-Printer-Attributes ipp://192.168.1.2:631/

D [23/Sep/2008:19:50:47 +1200] Get-Printer-Attributes client-error-not-found: The printer or class was not found.

D [23/Sep/2008:19:50:47 +1200] cupsdProcessIPPRequest: 8 status_code=406 (client-error-not-found)

If you find this occurring the most likely reason is that the OS X machine doesn't know the special URL of the printer, and is trying the default. When setting up the printer, fill in the "Location" field to point to the printer, including the printer name (as CUPS understands it). For example, if CUPS thinks the printer is called "Brother_HL-2040_series_USB_1" then the location would need to be similar to:

192.168.1.2:631/printers/Brother_HL-2040_series_USB_1

(with the right IP address and printer name of course). Remember to view the CUPS online page (by going to http://printerserver:631/printers) to find the name CUPS has given the printer.

Update - 6th Oct 2008

Recently I had the fun of testing out my notes above as I had to setup the print system again. This second time OS X was able to find the printer immediately with no problem under the Default Browser (this may be because I had subsequently installed the printer drivers from the Brother site). While OS X was able to find the printer though, it was failing to actually print. The cause turned out to be due to the server now giving access. This was finally solved by including the following section in the CUPS printer configuration file:

<Location /printers/HL-2040_series>
Order Deny,Allow
Deny From All
Allow localhost
Allow 192.168.1.*
AuthType None
</Location>

What is this?

2008-08-25T06:18:00.000-07:00

This blog is written by Jamie Love, a long time software developer based in New Zealand.

This blog is for the thoughts and ideas that come from my personal experiences in software development in New Zealand. The focus is on software development, telecommunications, being an entrepreneur and data visualization.