Flying (Lesson 2)

I recently started taking flying lessons with the Sydney Flying Club based out at Bankstown.

I had been a bit anxious about being able to fly this week as Sydney has been blanked by slightly unpleasant weather lately – we had a storm forecast for Friday which looked like it would hit around 3-5pm.  (2-3pm is usually when I go up).

I managed to get a booking in for 11:30am, so I hiked out to Bankstown, and fortunately, the weather was good enough.

The aerodrome was surprisingly busy. Plenty of helicopter traffic taxing around, and a private jet (looked like an Embrarer, but I don’t know their models well enough to identify which one) taxied past us as we walked to the Piper Warrior I was booked in for (VH-SFM).

(To help follow the rest of the post, here’s a map of Bankstown Airport)

SFM was parked in the flight club’s field parking between taxiways L and P.   After completing the preflight checks to the satisfaction of the instructor, we discovered that SFM’s flaps weren’t retracting when we dropped the flaps lever.

Thinking it’d correct itself under airflow, we set out via taxiway P (near the junction to L) upon which point the Instructor took control, set the brakes and tried to throttle up to force the flaps up with the prop wash.

When that didn’t work, we parked again, stopped the engine, un-jammed the flaps – confirmed that they were operating correctly again (they were, mostly – at least they retracted now under airflow force), and set off once again towards the runways.

I taxied all the way from parking to the A run-up bay, stopping to give way to other aircraft leaving the controlled tarmac for the apron,  ran the run-up checks, then taxied up to the A8 holding for 29R.

My instructor, satisfied with my taxing, instructed me that I’d be conducting the takeoff today, let me line up on the runway, and away we went.

Lesson itself was fairly uneventful – “Straight and Level” – all about maintaining level flight at different speeds/attitudes.

The flight back to the airfield was interesting however – by the time we returned from the training area, the winds had changed and the temperature had increased sharply – we were cleared to land on 11L. The instructor takes over from the reporting point, and everything is in order for a nice landing.

He sets up, deploys the flaps and bleeds off speed for final — and we just float there. Turns out that’s there’s a massive updraft freshly developed in front of 11L and because we hadn’t established our descent rate quickly enough, we’re now gently floating down – with our estimated touchdown point leaving us only 40% of the runway. He calls a go around and we re-enter circuit, and are given immediate clearance to land again (phew).

Fortunately attempt two went better and we touched down happily on the first half of the runway.  He taxis off the runway, hands back control, and I taxi us back to the field parking, where I brought the Piper in to park smoothly.

Jill

Japan Music Festival at the Roller Den, Sydney

I tagged along to the JMF on a whim, partially as a fan of JRock/JPop – and partially as a photographer looking for somewhere to practice my camera-craft during my off-season.

It was a great show.  Ended up buying Jill’s and kaimokujisho’s albums.  Should have picked up Sparky’s album too, but I was suffering pretty badly from the volume levels and was trying to make it out without spending all of my cash.

Unfortunately for me, I made the amateur’s mistake of not taking earplugs with me – my ears finally stopped ringing 3 days later.

It was a great first outing for my new second-hand 70-200 2.8 VR, which performed admirably in the conditions.  I had also completely forgotten some of the crazy with the D3 and the space remaining counter, so I thought I was running out of space by the end of kaimokujisho’s set – many poor (but potentially salvageable) bursts got deleted, and I thought I had significantly less space during 101A’s set than I actually did, so shot hyper conservatively…  then I discovered about 3 minutes into Jill’s set I had a whole memory card free.  I’ll try to remember about the damned counter next time.

A special kind of Hell…

So, I thought that ActiveRecord transaction support was finally reasonably mature and usable in Rails 4.

How wrong I was.

One of my biggest complaints with ActiveRecord has been that there was no way to set transaction isolation for a given transaction. This got addressed in Rails 4. Unfortunately nothing else appears to have been fixed.

The Problem

The problems remaining in ActiveRecord surrounding transactions:

  • No connector agnostic handling of transaction collisions/forced rollbacks.
  • No visibility of transaction isolation level inside a transaction
  • No automatic restart facilities for transactions.

No connector agnostic handling of transaction collisions/forced rollbacks.

My last post surmised why this is a problem pretty well – you need to understand your specific connector well enough to understand precisely how it’s going to tell ActiveRecord that the transaction failed. This is incredibly non-portable as a result – ActiveRecord is supposed to hide database details, not expose them!

No visibility of transaction isolation level inside a transaction

Transaction isolation is nice and fine, but the way it interacts with transaction nesting is a pain in the ass.

If a new deeper transaction is requested in a nested set with equal or lower transaction isolation to the open transaction, it should just work as normal without changing the isolation level. Whilst I’m sure there’s plenty of arguments to do with performance as to why this is suboptimal, but if you’re lowering isolation levels for ‘performance’, you probably shouldn’t be running those statements inside of a higher isolation transaction block.

I can’t even wrap this in since the isolation level information is discarded when the transaction is created – ActiveRecord makes no effort to remember what isolation level we’re running at.

This one is pretty important too since you can’t always see the implementation details of model methods at a glance and as such can’t tell if it’s usually a short transaction to ensure that it’s results are consistent.

No automatic restart facilities for transactions.

Not everybody realises that a transaction can fail because the database can’t guaranty it’s consistency, leading to the incorrect application of transactions by beginners.

Having an optional automatic restart facility would alleviate some of these problems as it would make it clear that it may be required in some circumstances to handle a transaction restart, and also allow users to get that behaviour with minimal effort.

It also facilities some improvements I have thought of…

A Proposal

I think we need to make the following changes to the transaction code:

  • Create an IsolationLevel class or comparison method so it’s possible to actually compare isolation levels against each other easily. As there is a clear heirachy of isolation with the standards based isolation levels, we should be able to compare them naturally in our code, and it should be possible to add any non-standard levels a driver offers.
  • Add an isolation_level attribute to the Connection.current_transaction objects so you can discover what isolation level is currently in effect
  • Add an optional restartable attribute to the transaction which defaults to false which indicates if the transaction can be auto-restarted. Add a optional max_retries attribute to specify how hard it should try.
  • Treat ALL transactions uniformly – at the moment the code differentiates a transaction started with an explicit isolation level parameter vs one without. You should be able to join transactions at the same isolation level that you desire at the very least, and there’s little harm in allowing transactions of lower isolation levels to join higher isolation level transactions.
  • Consider using the auto-restart facility, if enabled, to kick the isolation level on a transaction to a higher isolation level if required by a sub-transaction. This can be expensive if it’s happening a lot, but it’s better than some of the hoops required without it. In a lot of circumstances, however, the cost should be negligible as the query caches should help reduce the second execution costs.

Detecting Transaction Failures in Rails (with PostgreSQL)

So, Rails4 added support for setting the transaction isolation level on transactions. Something Rails has needed sorely for a long time.

Unfortunately nowhere is it documented how to correctly detect if a Transaction has failed during your Transaction block (vs any other kind of error, such as constraints failures).

The right way seems to be:

RetryLimit = 5 # set appropriately...

txn_retry_count = 0
begin
  Model.transaction(isolation: :serializable) do
    # do txn stuff here.
  end
rescue ActiveRecord::StatementInvalid => err
  if err.original_exception.is_a?(PG::TransactionRollback)
    txn_retry_count += 1
    if txn_retry_count < RetryLimit 
      retry
    else
      raise
    end
  else
    raise
  end
end

The transaction concurrency errors are all part of a specific family, which the current stable pg gem correctly reproduces in it’s exception heirachy. However, ActiveRecord captures the exception and raises it as a statement error, forcing you to unwrap it one layer in your code.

On Python and Pickles

Currently foremost in my mind has been my annoyances with Python.

My current gripes have been with pickle.

Rather than taking a conventional approach and devising a fixed protocol/markup for describing the objects and their state, they invented a small stack based machine which the serialisation library writes bytecode to drive in order to restore the object state.

If this sounds like overengineering, that’s because it is. It’s also overengineering that’s introduced potential security problems which are difficult to protect against.

Worse than this, rather than throwing out this mess and starting again when it was obvious that it wasn’t meeting their requirements, they just continued to extend it, introducing more opcodes.

Nevermind that when faced up against simpler serialisation approaches, such as state marshalling via JSON, it’s inevitably slower, and significantly more dangerous.

And then people like the celery project guys go off and make pickle the default marshalling format for their tools rather than defaulting to JSON (which they also support).

Last week, I got asked to assist with interpreting pickle data so we could peek into job data that had been queued with Celery. From Ruby.  The result was about 4 hours of swearing and a bit of Ruby coding to produce unpickle. I’ve since tidied it up a bit, written some more documentation, and published it (with permission from my manager of course).

For anybody else who ever has to face off against this ordeal, there’s enough documentation inside the python source tree (see Lib/pickletools.py and Lib/pickle.py) that you can build the pickle stack machine without having to read too much of the original source.  It also helps if you are familiar with Postscript as the pickle machine’s dictionary, tuple and list constructors work very similarly to Postscript’s array and dictionary constructs (right down to the use of a stack mark during construction).

LCA2012: Day -1: Arrival and Registration

So, I’m in Ballarat for LCA2012, courtesy of my employer.  I’ll be presenting on Tuesday along with David on some of the work I’ve done on a few tools for systems automation.

The flight was uneventful – although the landing was a bit rough…

The drive from the car rental place to Ballarat was fortunately uneventful, but plagued with a combination of idiots who insist on doing things the hard way (for crying out loud, don’t try to shoot past me on the left if I’m indicating a left merge already >_< ) and the fact the car’s speedo reads about 5-7kph under actual resulting in us doing about 92kph down the western highway when I thought we were doing 98kph (I have a 100kph limit still.  damned P-Plates).  The car is a newish Camry and is by far the newest thing I’ve actually driven any distance now – quite impressed in some ways, but disturbed by the lack of vision due to the body shape (pillars are huge >_< ) and the very light steering.

Fortunately we reached Ballarat and found the motel without troubles.

Registered at the University and picked up the schwag pack.  This year’s bag is a crumpler sack style thing, contianing: A catalyst branded screen cleaning cloth + case, Bulletproof Networks branded mousemat (looks like it might be faux leather or even real leather), an LCA branded keep cup, intel branded universal USB -> smartphone charger, DSD branded stubbieholder and careers spiel pamphlet, LCA t-shirt, LCA conference guide and a Freetronics LeoStick.

Met up with some people and went to Murphy’s – an Irish pub on the main drag – for dinner.  Had the Irish Pizza which was quite pleasant, and generally emptied my wallet on food. (Ouch).

All in all a good start to the week – tomorrow is miniconfs.

On opinions, beliefs and conflict…

Opinions.

Everybody seems to have one.  The problem is that everybody tends to have a different one.

From difference breeds the basis for conflict.

Scales of conflict can range from a gentle discussion about the differences in opinion to genocide and eugenics.  Fortunately people are rarely in a position to execute the most extreme of these, but it is all bred from the same source.

We like to think that modern humans are civilized and can discuss differences in opinion in a calm and polite manner.  We like to think that when we are discussing opinions that we can do so in a receptive matter.

The truth is that not all humans are as receptive or open minded as we like to think ourselves to be.

The more we believe an opinion, the more we are likely to defend it strongly.  The more we believe an opinion, the more likely we are to take offense when somebody else speaks to the contrary.  The more belief involved, the worse the following conflict will be.

Belief is orthogonal to rationality.  They may coincide on a particular topic because the belief is formed from rational thought, but just as frequently they will not.

We believe in freedom of expression.  We frequently forget that by expressing our beliefs we can be infringing upon the beliefs of another.

Should you choose to openly infringe on the beliefs of others, you should be prepared for the conflict that follows.  If your expression of your opinions leaves a larger majority feeling threatened, you may have quite the conflict to deal with.  Morality grants you no boons in this – if you cannot hold your ground in doing so, you shouldn’t step out so far.

This is not pointing blame.  This is not assigning culpability.  This is just the nature of opinions.

This is my opinion. It’s not right, wrong, better, worse, or any combination or permutation of the above.  It is simply mine.

I respect that you have an opinion.  Maybe it aligns, maybe it doesn’t.

If you truly respect freedom of expression, you will accept this, and you will accept my reasoning.  You don’t have to like it, you don’t have to agree with it.  You are perfectly entitled to disagree.

I am just as guilty of being inflamed when people challenge this belief – more so that through challenging it, they are merely reinforcing it. It leaves me frustrated as frequently I see the other party unable to put aside their personal stake or fail to disassociate from the initial argument, and in doing so, they fail to see this pattern emerging in the conflict that follows and instead try to insist that their beliefs are somehow more “right” or “correct” than this.

Philosophy aside, understand that this is also why I don’t approve of contentious minorities trying to ‘fight the good fight’ against the better part of incumbent belief.  I may agree with your underlying opinion, but to draw such attention down upon yourself when you’re ill prepared to survive it is fool-hearty.

If you were to ask me how I would change the world – if I genuinely felt that an incumbent belief required ‘intervention’ in order to correct some injustice I would most likely answer the following:

Recruit quietly.  Work silently to your goals through example.  Unless you have overwhelming support, do not challenge the old ways head-on, but work to slowly build a foundation and support for the new without challenging the old.  Eventually you’ll find the strength required to plow forward and supplant the old, or you will have reached a point where opinion can sway to your beliefs on it’s own.

Ubisoft: When Offline Isn’t Offline

Given the furor that Ubisoft has been experiencing over their DRM methods, you’d think they would have learnt to back off.

Unfortunately they haven’t.

For all the ‘offline mode’ support Anno2070 claims to have, once you’ve performed the one-time Tages authentication check, you also need to create an ubi.com account, register your game key against it (which invalidates the key), and log in at least once.

I wanted an offline mode to avoid this sort of shit treatment.

No longer will I put up with this sort of crap.

I don’t want to create and maintain accounts for every publisher and/or every game.

I don’t want to have to worry about losing those account details when I put the game down and come back to it in 6 months.

But most of all, I feel like the publishers are intentionally trying to violate my privacy by tracking my game actions and forcing me into signing into an online service I didn’t want to play what’s fundamentally a single player experience.

No longer.  I am now actively refusing to buy any Ubisoft games, irrespective of platform, developer or quality.  Unless they demonstrate a healthy dose of reality, I doubt they’ll be coming off of it again.

And I will be encouraging my friends and peers to do the same.