Category Archives: Code

Nine newspapers: one for all

Nine Entertainment’s major Australian newspapers, The Age, Sydney Morning Herald, and Brisbane Times, have a soft paywall. You can get some articles for free; some are only for paying customers; and some fall somewhere in between, with them granting complimentary access to a few, before asking for money.

This post is not to tell you about various ways of bypassing the paywall if you don’t have a subscription.

But if you have a subscription to one of these, then you can get access to the other two.

I subscribe to The Age, but sometimes hit the paywall when trying to read SMH or BT articles.

In fact sometimes the links within Age articles are sloppy, and point to SMH. Ditto some of the social media shares I see that are from people who I know to be Age readers. I think there’s something dodgy in the app or web site that sometimes spits out an SMH link.

Then I noticed the articles on each site are replicated to the others. If there’s an smh.com.au/whatever/article URL that I can’t read, simply changing it to theage.com.au/whatever/article works. Same with brisbanetimes.

Not too hard to do this by hand in the browser, but it can also be done automatically with a browser plugin such as Redirector.

I wanted it to preserve the SMH or BT home pages, so I can see what the Sydney or Brisbane headlines are. So I added an exclusion for that.

It just about works, with the caveat that if browsing the SMH or BT home page, you need to right-click and open link in a new tab for Redirector to kick-in with the modified Age URL.

Import the JSON config I’ve listed below, or write your own.

{
    "createdBy": "Redirector v3.5.3",
    "createdAt": "2024-11-12T04:20:41.494Z",
    "redirects": [
        {
            "description": "SMH to Age",
            "exampleUrl": "https://www.smh.com.au/technology/shameful-tech-council-facing-questions-over-richard-white-saga-20241025-p5kla6.html",
            "exampleResult": "https://www.theage.com.au/technology/shameful-tech-council-facing-questions-over-richard-white-saga-20241025-p5kla6.html",
            "error": null,
            "includePattern": "https://www.smh.com.au/*",
            "excludePattern": "https://www.smh.com.au/",
            "patternDesc": "",
            "redirectUrl": "https://www.theage.com.au/$1",
            "patternType": "W",
            "processMatches": "noProcessing",
            "disabled": false,
            "grouped": false,
            "appliesTo": [
                "main_frame"
            ]
        },
        {
            "description": "Brisbane Times to Age",
            "exampleUrl": "https://www.brisbanetimes.com.au/business/companies/abc-to-slash-dozens-of-jobs-ahead-of-restructure-20230615-p5dgoe.html",
            "exampleResult": "https://www.theage.com.au/business/companies/abc-to-slash-dozens-of-jobs-ahead-of-restructure-20230615-p5dgoe.html",
            "error": null,
            "includePattern": "https://www.brisbanetimes.com.au/*",
            "excludePattern": "https://www.brisbanetimes.com.au/",
            "patternDesc": "",
            "redirectUrl": "https://www.theage.com.au/$1",
            "patternType": "W",
            "processMatches": "noProcessing",
            "disabled": false,
            "grouped": false,
            "appliesTo": [
                "main_frame"
            ]
        }
    ]
}

Hopefully this is useful to others who subscribe to one of these sites.

Programatic submission of Australia Post’s CN23 customs form

A number of major international destinations of packages now will only accept packages with electronic CN23 customs declaration. Normally, you’d do this by rocking up to the Post Office with your pre-addressed parcel, filling in a CN23 paper form, and have that transcribed into Australia Post’s computer system by the postal worker behind the counter. You can elect to receive SMS notifications of change of status (landed, delivered, etc) for 50c.

Australia Post also allows you to fill in the appropriate details on their website; if you do this, then you get a QR code sent to you via SMS (free) and email (free) which the postal worker scans in and all the details (your name and address, destination name and address, contents, etc) are attached to your package’s details without any error-prone re-keying. The downside of going down this path is the dismal website Aussie Post provides, a JavaScript heavy, painfully slow dog of a site that doesn’t cache your own address.

Once the QR code is scanned, and the postal worker checks everything with you, they’ll print out the CN23, get you to sign it , and then it gets attached to your parcel. Because the To and From addresses are on the CN23 form (and those details are in electronic form, associated with the barcode for the package), it’s perfectly acceptable to present an unaddressed package to the post office (make sure you can tell which package is which, if you go down this route).

One thing you need to be aware of: Australia Post hasn’t heard of Unicode. You absolutely can’t use any characters not in the ASCII character set, and even then a very limited range of them. Certain fields allow some characters, which in turn aren’t allowed in other fields.

One of the fields you can supply is the HS tariff code, which is an international standard group of codes to describe “stuff” – the Harmonised System Tariff code. The sourcecode below uses the code for “Toy, plastic construction” – you should use the code for what you’re actually sending. You can specify multiple HS codes. Dollar values are in decimal dollars, weights are in decimal kilograms.

After calling the Australia Post website with your customs declaration, it returns to you a base-64 encoded PNG of the QR code to present at the counter, and a base-64 encoded PDF of the CN23 form – there’s no point printing this out, because it’s not paid for yet; let the Post Office print it out with the postage on it. You’ll also get the PNG via email and SMS (free).

Here’s some Python to make this submission:


    AP_session = requests.Session()
    jsonFormData =  {"customDeclaration":{
      "label":{"source":"AEM","postagePaidIndicator":False,"eadIndicator":False},
      "parcelCharacteristics":{
        "productClassification":11,
        "dangerousGoodsIndicator":False,
        "returnInstructions":"Return By Most Economical Route",
        "confirmationMobileNumber":"0411111111",
        "content":[{
          "content":"HS traffic code name for your stuff",
          "contentQuantity":1,
          "contentUnitValue":subtotal,
          "totalContentValue":subtotal,
          "contentWeight":int(order["total_weight"])/1000,
          "hsTariff":"95030039",
          "contentCountryOfOrigin":"DK"
          }],
        "totalConsignmentValue":subtotal},
      "senderAddress":{"firstName":"Josh","lastName":"FromGeekrant.org",
        "addressLine":["11 Example St"],"suburb":"YourSuburbName","state":"VIC",
        "postcode":"3000","email":"addr@example.com",
        "phone":"0411111111","smsConfirmation":False,"countryCode":"AU"},
      "receiverAddress":{"firstName":CustomsString(order["label_address_name_first"]),
        "lastName":CustomsString(order["label_address_name_last"]),
        "countryCode":order["label_address_two_char_country_code"],
        "addressLine":CustomsAddress(order),
        "suburb":CustomsString(order["label_address_city"]),
        "state":CustomsString(order["label_address_state"]),
        "postcode":CustomsString(order["label_address_postal_code"]),
        "email":order["buyer_email"]}
    }}

    stopact = {"jsonFormData":json.dumps(jsonFormData) }
    result = AP_session.post(url='https://auspost.com.au/bin/form/stopact', 
      data=stopact, timeout=2)
    response = json.loads(result.text)
    result.raise_for_status()
    filename = "{}-customsQRcode.png".format(orderid)
    with open(filename, "wb") as fh:
      fh.write(base64.b64decode(response['qrCode']))
    filename = "{}-CN23.pdf".format(orderid)
    with open(filename, "wb") as fh:
      fh.write(base64.b64decode(response['label']))

Setting up a public facing webserver behind a Netcomm NF18ACV

Note: this will move your Broadband Router’s web-configuration to an unexpected port :8080, instead of the :80 your browser expects.

Navigate to Management | Access Control | Services then disable the WAN side HTTP service (why would you even expose this?), change the port for the LAN side to the Alternate HTTP port of 8080, and hit Apply/Save.

Navigate to Advanced Setup | NAT | Virtual Servers and hit Add. Select the correct interface, fill in the other details including the Web Server’s LAN address, ensure you’ve got Status: Enabled for the port forwarding, and hit Apply/Save.

Run up a trial HTTP server using something like
sudo python -m SimpleHTTPServer 80
and check for access from outside. Kill the server, because that isn’t safe for production use.

How to block notification requests in Chrome

Oh the irony.

Anyway, the steps:

  1. Go to Chrome Preferences
  2. Advanced
  3. Site settings (or “Content settings”)
  4. Notifications
  5. Click the “Ask before sending (recommended)” switch off. It should then say “Blocked”

Yeah it’s a counter-intuitive caption on that button. It implies that switching it off will just drown you in a sea of unwanted notifications without asking you.

But it seems to work.

Update 2023: It’s moved a bit, so now try:

  1. Chrome Settings
  2. Search for Site Settings
  3. Default behaviour: Choose “Don’t allow sites to send notifications”

If you use Firefox or Safari, this How To Geek article covers those. The article notes that Edge doesn’t currently have an option to turn them off for good.

Where do you wake up from a bed in Minecraft?

After issuing many /time set night commands, I can tell you the waking-location algorithm for Minecraft. This presumably also affects your spawn point.

Two locations are checked, and if they fail to select an acceptable location the pillow-location is used regardless of consequences of picking this location. An acceptable location is on the same level as the bed, and has two transparent-non-solid blocks above it (i.e. you will be standing next to the bed without your head or body embedded in something that’s killing you).

The process is the same for each of the two locations:

Sweep x-1 to x+1:
  Sweep z-1 to z+1:
    if the location is acceptable, we're done

The locations are checked in the order: pillow-part-of-bed, non-pillow-part-of-bed. The effect is:
From the Minecraft wiki:

For a bed to be usable as a spawn point, the player must be able to stand next to the bed at the same level as it. There must be a solid block at the same ‘floor’ level as the bed, with 2 transparent blocks of space (for example, air) for the player to stand in, in one of the ten blocks that surround the bed. It doesn’t matter if the bed itself has blocks above it.

How to to install the Crypt::Eksblowfish::Bcrypt module, and Crypt::Random

Have you gotten the error
Can't locate Crypt/Eksblowfish/Bcrypt.pm in @INC (you may need to install the Crypt::Eksblowfish::Bcrypt module)
and wondered what to do? Wonder no more!

sudo apt install libcrypt-eksblowfish-perl

and perhaps

sudo apt install libdigest-bcrypt-perl

What about
Can't locate Crypt/Random.pm in @INC (you may need to install the Crypt::Random module)
Easy!

sudo apt install unzip make gcc
wget http://search.cpan.org/CPAN/authors/id/I/IL/ILYAZ/modules/Math-Pari-2.01080900.zip
cd Math-Pari-2.01080900/
perl Makefile.PL
sed -i 's/CLK_TCK/CLOCKS_PER_SEC/g' pari-2.1.7/src/language/init.c
make
make test
sudo make install
cd ..
wget http://search.cpan.org/CPAN/authors/id/V/VI/VIPUL/Crypt-Random-1.25.tar.gz
tar zxvf Crypt-Random-1.25.tar.gz
cd Crypt-Rando1.25.tar
perl Makefile.PL

Easy! Only takes a few hours if you don’t know what you’re doing.

Git: prune those old branches

TortoiseGit’s right-click sometimes appears to freeze.

It can be caused by too many old branches hanging around.

Here’s how to prune the ones that have been merged to master:

git branch --merged master |grep -v master | xargs git branch -d

Obviously you’ll want to make sure your master branch is up to date.

Thanks to my colleague UA for finding that one.

TortoiseGit claims commit message is empty

I was wondering why TortoiseGit* was complaining that my commits had no message in them.

“Aborting commit due to empty commit message.”

Turns out the new version by default strips out messages starting with a # character.

As detailed in this issue report, the over-ride is in Settings / Dialogs 2.

TortoiseGit options

*yeah, I’m not a hardcore Git user, though I do prefer Git Bash on Windows for some operations

Flickr’s new HTML code embedding – how to remove the header and footer

Flickr has altered its default embed HTML to include a header and footer, which includes Flickr branding and the title of the picture.

PT in the Sense8 titles 01

Sometimes I suppose this is okay, but sometimes I just want the picture.

Fortunately it seems to be relatively easy to get rid of. In the example above:

<a data-flickr-embed="true" data-header="true" data-footer="true" href="https://www.flickr.com/photos/danielbowen/19038778583/in/dateposted/" title="PT in the Sense8 titles 01"><img src="https://farm1.staticflickr.com/313/19038778583_3149e7e01a.jpg" width="500" height="282" alt="PT in the Sense8 titles 01"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script>

…remove the data-flickr-embed, data-header, and data-footer attributes of the a href, and remove the script tags, like this:

<a href="https://www.flickr.com/photos/danielbowen/19038778583/in/dateposted/" title="PT in the Sense8 titles 01"><img src="https://farm1.staticflickr.com/313/19038778583_3149e7e01a.jpg" width="500" height="282" alt="PT in the Sense8 titles 01"></a>

The result should be just the photo, with the usual linking back to Flickr.

PT in the Sense8 titles 01

It’d be nice if they made this a built-in option when generating the HTML code.

Of course, it also makes me ponder if I should be finding another photo host.

Update 2015-07-20: They seem to have modified their default embedding code a bit so the branding and picture details now only appear over the photo when you mouse over it. Not so objectionable.

PT in the Sense8 titles 01

Flickr’s modified code now excludes data-header="true" data-footer="true" which presumably added the header and footer.

COM+ calling multiple .Net versions risks blowing the memory limit

From the Things I’ve Found That Might Affect Other People But There’s Not Much Detail In Google About It department

Not that I have much to add to the sum total of published knowledge on this, but our situation is we have a COM object (legacy code) calling .Net 2 objects (not-quite-as-old legacy code).

We’re upgrading it all to .Net 4.5, but the .Net stuff is an easier change, so we’ve been doing that first, naturally in a carefully planned, staged manner.

It turns out that COM+ objects have a 1 Gb memory limit.

And it appears that when you have them calling a mix of .Net 2 and .Net 4 objects, the overhead of being able to call two different .Net Framework versions becomes an issue.

At least that’s our theory based on what we’re seeing — looking at ProcessExplorer showed about a dozen assemblies loaded for each version of .Net.

One blog post I read suggested a 600-800 Mb overhead for each .Net version. I can’t confirm that, but certainly we’ve had more Out Of Memory exceptions than we expected.

We’ve resolved it by going back to our .Net 2 code (so only one version is held within the COM+ process at once), and we’ll do the big switch all at once. Hopefully that’ll resolve it.

The COM object will also be switched too of course, and once that’s done, running everything in .Net 4, plus a move from 32-bit to 64-bit machines, apparently will lift that 1 Gb limit to 8 Tb!

Cool!

SQL Server paranoia mode

This post is for Tony!

Worried about accidentally overwriting critical data with a typo in your database commands?

SQL Server Management Studio has an option for that: SET IMPLICIT TRANSACTIONS, which you’ll find (in 2008 / 2008 R2 at least) under Tools / Options / Query Execution / SQL Server / ANSI.

SQL Server implicit transactions option

Anything you do (from the next query window you open) will automatically be in a transaction, so you can ROLLBACK if you realise you’ve done the wrong thing.

Be warned, you’ll need to get into the habit of manually COMMITting everything. Don’t be tempted to just add the COMMIT at the bottom of your query… that would defeat the purpose.

It’ll prompt you do to so if you close a query window without having done a ROLLBACK or COMMIT. It can get a little irritating, but knowing you can’t accidentally trash all your data may give you piece of mind in return.