Random Neural Firings

Well, it’s only been, what, two years?  I guess I’ll just claim that I’m *so* busy that I just can’t find the time to write blog posts and I don’t use the Twitter or the Facebooks, so to all my devoted readers who have been waiting with baited breath these last two years…I’m back.

I’ll write more later.  (I know, famous last words, right?)

The boys and I are working on some new music. These are still very early stage demos, so they’re pretty rough. What can you expect from original music produced at home?

Kyle comes up with most of the musical concepts. He’s about as prolific a songwriter as I’ve ever met. So these are his babies. Zach and I just fill in the texture with other musical stylings to go along with Kyle’s original concepts.

The first tune, This Love, is our latest and probably most complete as far as song structure and direction, though it’s still far from finished as to production. That’s Kyle on the vocals (obviously) and rhythm guitar. I’m on the drums and bass. Zach hasn’t made an appearance on this tune yet.

Next is our least developed tune, Drop A. It’s just an instrumental jam at the moment based on some wild tuning experimentation. As the title suggests, the low E is dropped all the way to A. Kyle is on the acoustic rhythm. Zach on melodic lead guitar exploration. I’m on bass. Drums are actually a loop in this one, but I have a drum line that I’ll eventually lay down for it.

Finally, Utah High is also pretty near completion as to structure, but far from being production ready. Hey, it’s a demo. Kyle once again on vocals and acoustic rhythm. Zach’s on electric rhythm and lead. I’m on bass and drums.

Enjoy.

My time at Overstock.com is over…thankfully. I have been looking to make a move for some time anyway, but I didn’t want to pull the rug out from under them. Ironically, that’s exactly what they ended up doing to me, so let this be a lesson to all employees everywhere: there is no such thing as “company loyalty.” At best, that’s a one-way street with the employee being expected to show loyalty to the company. It never goes the other direction. Any company will saw you off in a heart-beat if it means saving a few dollars for the executives bonuses. So don’t buy into this loyalty B.S. It doesn’t exist.

I’m actually not even upset about the termination of the contract. As I said, I was looking to make a move anyway. Overstock makes a lot of money, but you’d think a company that can generate in excess of $200 million in revenue in a quarter could figure out how to not still be left with a $3 million loss. Not so in this case, and now, having been inside the organization for a few months, I understand why.

When I first started there, I thought the culture might be different, better than most large corporations. The average employees have pretty good exposure and access to executive management, which you would think would lead to the upper managers being better able to do their jobs–namely, to clear away the barriers to success that the actual workers face. In the end, that’s the only role of management: to make sure the people actually doing the work can do it free from burdens so they can actually be successful.

Sadly, in Overstock’s case, this isn’t what is happening. I guess I can’t speak intelligently on the entire company because I only worked in one particular segment. But due to the physical working environment–rows of desk clusters with very little separation between one team and then next–I did have a pretty good look at the operations of at least one entire floor.

In the case of the Social Media team, the biggest issue is leadership. There is none. Upper management had designated two team leads–a developer leader and a project leader. Neither of them was actually a leader by any stretch. Leadership requires a certain disposition and personality, and a certain thickness of skin. All of these things were conspicuously absent in both.

The developer leader, a nice guy to be sure, seemed to always be clinging to the frayed ends of his sanity. He seemed overwhelmed from day one by the burden placed on him by the expectations for the team. That’s where the problems started. Neither he nor the project lead seemed capable of telling the CEO “no” when it came to certain projects. The CEO had a personal interest in the developments of the Social Media team so he made a point to stop in on a regular basis and get updates on all his pet projects. Rather than telling him that adequate time was needed to plan for these projects, after which they could provide a development time line, they would just “yes” him and make him think that everything he wanted to have done could be done when he wanted it. The end result was that the CEO had little confidence that anything would be done ever by that team, because all he ever got was yeses followed shortly by excuses as to why it didn’t get finished on time.

Adding to the trouble was the fact that the developer leader, obviously overwhelmed by the responsibility of unrealistic expectations and constantly battling against the demons of some previous failure that caused him to fear beyond reason any future failures, had no sense of delegation or organization. He could make assignments to his team; but too often those assignments would conflict with each other in various ways. The team would end up all working on the same project, theoretically on different pieces of the project; but things were so poorly managed and organized that those pieces would invariably step on the toes of the other pieces, delaying completion and launch of the finished project often by weeks.

Even worse, no one on the team (which included developers, project managers, QA and IT personnel) actually understood the proper procedures that should be followed to take a project from concept through development, QA and finally launch. It was me–the new guy–who finally had to lose patience and go get the answers from the sources. This was the beginning of the end for me. The team had been engaged in endless bickering about how to actually get a project into a proper test environment for QA and no one knew the answer, yet they kept bickering about it. Finally I just left and got the answer from the QA department. It took all of 10 minutes to get the answers, which I relayed to the team, much to their amazement.

A bigger problem than the developer leader, though, was the project leader. Also a nice guy, I honestly, even now, have no idea what this guy did on the team other than meet with his superiors to give status reports and take requests (which, of course, he didn’t know how to say “no” to). I was on this team for some four months, give or take, and in that time, I saw him do a little graphic design work during my first week, and from then on, every time I saw his computer he was just browsing on Facebook or LinkedIn or some other social networking, web 2.0 site (apparently doing market research or some such B.S.). He had no technical background at all, yet he was at the head of the team that needed to be on the cutting edge of web technology. This is probably the biggest problem I’ve run into in countless companies that I’ve consulted for: somehow or another, there seems to always be some guy that no one knows what to do with who ends up in the nebulous “project lead” position, trying to manage technical operations and resources without any clue at all what kinds of efforts are required to accomplish tasks. You can see how problematic this is, to have non-technical people making promises for delivery of technical projects without any consultation among the technical people.

Making matters worse, this guy reported directly to one of the company Vice Presidents. The VP, a woman in this case, was also non-technical and totally uninterested in what the team was doing, yet the team had been placed under her supervision, I can only guess because she was the only one who wouldn’t give the CEO crap about it. (See, the rest of the company believes the Social Media team to be a waste of resources and one of the CEO’s little “pet projects” that no one else takes seriously.) So the team operated in relative invisibility. The direct supervisor at the executive level didn’t care about the team and based her operational decisions entirely on the information provided by the project lead, the proverbial “yes man,” who had no real understanding of what the team was up against. The CEO got some of his information from his VP and some from direct interaction with the team, but in neither case was his exposure sufficient to give him an accurate picture of the team’s barriers. So the team labored along, incapable of any significant success for reasons that were as plain as Amish women, but the decision-makers who could correct the situation couldn’t be bothered to look.

(Note: the “Amish women” comment isn’t an insult. The Amish joy in plainness. It’s part of their belief system, which is why the comparison works.)

Bottom line: the team was doomed to failure from the minute it was formed.

In the time that I was there we were given four major projects. During that time I worked on all four of them, but was never allowed to actually finish any of them. In every case, just as I got finished with the bulk of the code (which I did in every case), the project was assigned to someone else and I was given a new one.

It started with the so-called “Patrick widget” which I finished in about a week, for the most part. All of the major functionality was finished in that time, but before I could finish up the smaller house-keeping issues, I was re-assigned to work on the “forums” project. That lasted for a few days before I was re-assigned again to work on the “video” project, which was basically Overstock’s version of YouTube. Some of that code was already done, but I finished the rest of it, again to the point that the major functionality was complete and we were working through little bug fixes and enhancements. Then that project got split into two: a “partner” version, and a “public” version. The partner version took the focus, with the public on the shelf until further notice. This project was the worst, because it was on this project that the dev leader assigned everyone some piece of it and with no coding standards or common code libraries to work from, they all conflicted with each other. Just before that project was finished, I was taken off of it again and placed back on the forums. Fortunately I was alone on forums and was able to complete that one, again for the most part, without interruption from others. It took forever thanks to phpBB’s crap code (as I have described in previous posts on this blog), but ultimately I got it through the major barriers, just far enough to be taken off of it again, this time for good as my contract was terminated.

In a nutshell, any successes that team has had in the past few months is directly due and thanks to me and me alone. There is no way that they would have gotten even the first of those projects done were it not for my involvement. Incidentally, of all those projects, only one part of one of them has been launched so far: the partner video. The rest are still languishing in mostly-finished status due to the total mismanagement of these two “leaders.”

Needless to say, my departure comes none too soon. I’ve had my fill of Overstock.com.

In our first battle, we were simply trying to understand what needed to happen and where in order to produce a valid authenticated user. Really we didn’t even get that far; we simply identified how the modular authentication system of phpBB3 is supposed to work. We really didn’t get into the nitty-gritty of phpBB’s execution.

If we’re going to try to track our “quantum particles” (i.e. execution…see the series opener for explanation) through that nebula, the place to start is with the User Control Panel (ucp.php). That is, after all, where most battles take place, or at least where they begin.

So, from the top…

The script opens with several definitions and includes that contain more definitions. We’re not going to concern ourselves much with those. Just note that those lines appear in most of the public-facing scripts in phpBB.

In fact, we really don’t care about anything until we get to the session_begin() function. I went into some detail about this function in the previous post, so I won’t go into it again here. We’ll get into it later when we get back to the discussion of Overstock’s unique needs and how I was finally able to meet them.

After the session is opened, phpBB needs to detemine the permissions held by the user. That’s done by a call to the $auth->acl() function, which takes an array of user data as it’s argument. The user data array ($user->data) exists by virtue of the session that was previously opened for the user ($user). As that user object is constructed, user permissions are either queried from the database for that particular user (if a record exists) or they are queried from a “guest” account to allow the user to browse around the forums as a guest as allowed by the forum permissions (set by admin; outside the scope of this discussion).

Now that the session is created and the permissions set, the next thing we care about is $mode. You may have noticed a few lines up the line that defines the $mode variable by a call to the request_var function. I’ve never taken the time to read through this function, but my cursory observation and the obvious implication of the name lead me to deduce that this is phpBB’s wrapped method of extracting values from other places (such as REQUEST arrays). In this particular case, the value being extracted comes from the URL string, so it is a GET parameter called mode. If the user is not logged in, the value of this parameter will be login.

So, following through ucp.php, we come to a switch statement that is testing the value of $mode. For now, we only care about the block that executes if the value of $mode is “login.”

The original phpBB code checks to see if the user if registered. If so, the user is redirected to the index page; otherwise, the login_box() function is called, which (you guessed it) renders the phpBB login box. All well and good under normal circumstances, but in Overstock’s case, this wasn’t the desired behavior. As I mentioned in the first post, Overstock wanted to bypass the standard phpBB login process and replace it with their own “universal login” procedure. So rather than using the standard phpBB login screen, the desired behavior was for the system to check for an existing universal login session (certain cookie values), and if it existed, automatically log the user into phpBB; otherwise, redirect the user to Overstock’s univseral login screen, which would then send the user back to the referring page upon successful authentication. Once returned from the universal login, the user needed to be authenticated and logged into phpBB seamlessly.

Sounds simple enough; however there is one further complicating wrinkle that needed to be handled. In the event that the user had never used the Overstock forums before (i.e. didn’t have an existing phpBB account), the login procedure needed to create the phpBB account for them. So, effectively, the user would have two accounts–one universal login account for Overstock sites, and one phpBB account–that needed to be seamlessly synchronized. The phpBB account would be used for phpBB authentication and permissions, but the user’s experience has to act as though everything is controlled by the universal login account.

So back to the UCP.

To accomplish the desired effects, I took out all of phpBB’s original login code and replaced it with an if/else procedure that tests for a valid universal login session (ID). If found, the username and password for the user’s phpBB account are determined (I’m keeping the details of this procedure to myself for security reasons). Finally, it executes the $auth->login() function, passing in the username and password. If successful, it redirects to the index page and phpBB functions as normal from there with the user fully logged in.

So there it is. Mission accomplished…sort of. We’re now able to successfully authenticate a user in phpBB using an external authentication procedure. Of course, that assumes the user already has an existing phpBB account that corresponds to their universal login account.

But what if they don’t? How do we iron out that particular wrinkle?

Read on. We’re winning battles, but the war is still far from over.

As I mentioned in the introductory post, the first big battle to win against phpBB3 is understanding the execution path that is followed for simple operations like presenting the forum lists, login screens, etc. Without question, the bulk of time and trouble was related to this.

Again, credit to Zend for making it possible to even follow that path, since without it, I would be left to a lot of guessing and just reading of code.

<tangent>Let me just interject at this point that this is one of the most annoying tendencies of Open Source software developers: they make the assumption that since they are complete geeks who would just as soon read code as a good novel, that everyone else in the development world should do the same. I think this is more arrogance than laziness or misunderstanding in many cases; it’s their way of saying, “if you’re not willing to read every line of code, you’re not a real developer like I am.” I can read code just as well as anyone, but I’m actually a successful professional developer, so my time is much better spent in worthy pursuits like actually building things and is too valuable to be wasted reading every line of someone else’s code to try to decode their mental dysfunction so I can understand how and why they have done things the way they have. Documentation, geeks!</tangent>

Theoretically, phpBB3 supports a modular authentication model, so by setting a particular value in the database that tells phpBB what module to use, and then writing the functions for that module, you’re able to integrate your own custom auth code into phpBB3 (I’m not sure if this was also available in previous versions of phpBB). In Overstock’s case, authentication had to be customized, so step number one was to indicate what the name of the authentication module is. This is done by making an entry into the phpbb_config table (assuming your table prefix is phpbb_). Normally this would be done through the phpBB admin control panel, but for the sake of really understanding the innards, I’m going to refer to the actual database table and column(s).

The table phpbb_config has three columns defined:

• config_name
• config_value
• is_dynamic

This structure is intended to allow for configuration flexibility. When you identify an authentication module using the admin control panel, a new record is entered into phpbb_config with the values assigned as follows:

• auth_method
• [the name of your auth module]
• 0

Once this is set, when phpBB authentication executes, rather than using one of its own auth modules, it will use the one named in this record. Sounds simple enough, but there is one important thing to remember:

No matter what name you assign to your authentication module, phpBB expects the file name to be prepended with auth_.

So, for example, if you specify that your authentication module name is custom, phpBB will expect that the authentication functions for your module are defined in a file called auth_custom.php (assuming the extension you’re using is .php). So, in order to use your custom functions, you have to create a file called auth_custom.php and store it in the includes/auth directory of your phpBB installation.

Get used to that convention, _custom, because it represents the pattern that will persist throughout the customization effort. (Of course custom is what I’m using for my examples; you’ll need to replace that with your authentication module name wherever appropriate).

Now that you have the file in place, you have to create functions that phpBB will recognize. Once again, the authentication module comes into play. To log a user in, phpBB calls its login function, which is a member (method) of the auth class. The login function is just a wrapper for the login function found in the authentication module being used. If you follow into the auth class login() function, the first thing that happens (after initializing globals) is it calls in the appropriate authentication module and tries to execute the login_ function for that module. So for our example, it will look for a function called login_custom() inside of includes/auth/auth_custom.php. Once execution enters this function, you’re in complete control and you can pretty much do whatever you want. Of course, it won’t likely do you any good unless you excute at least the most fundamental of phpBB’s login procedures…i.e. logging in.

To log a user in and take advantange of phpBB’s session management, you have to pass a valid username and passsword into the login function:

$auth->login($username, $password)

This will return an array of results whether it succeeds or fails. One of the array elements is called status, which is what you’ll use to determine success. PhpBB makes extensive use of constants (defined in includes/constants.php), and in the native authentication code, phpBB will compare the status element of the array with the LOGIN_SUCCESS constant value. If they match, login was successful; otherwise, it failed. From there flow control is up to you to handle according to your needs.

It should be noted that this whole procedure is started by a call to phpBB’s session_begin() function. Open up almost any of phpBB’s files (i.e. ucp.php) and you’ll see a call to this function near the top. In phpBB, the session class actually extends the user class (or maybe it’s vice-versa…I forget), which is why you’ll see this function called as a method of the $user object:

$user->session_begin();

If you follow through the session_begin() function (defined in includes/session.php), you’ll find the code that implements your custom authentication module (as of writing, this code begins on line 270); however, the real code we have to look at first comes before that, on line 231.

At this point in execution, the system enters an if statemement related to the session. Don’t ask me what all the different variables being compared actually are or where they are initially defined. With phpBB’s extensive use of constants, globals, and it’s own session management (rather than native PHP sessions), it’s very difficult to know exactly what is being compared to what and where any of it comes from. The long and short of it is this, though: if there is a valid session available for this user, enter this block of code; otherwise, it jumps down to line 334 and starts a new session.

With the session either restored or created, execution can now proceed.

We’re not even close to finished yet, but at least we know where to start. Battle won. Chalk one up for the good guys.

Sadly, however, the war continues…

My current consulting gig is with for one of the largest retailers on the web, Overstock.com. The company is aggressively seeking to leverage social media and networking to improve attention and retention to their site–just like all companies that actually understand where the attention of the masses is at this particular moment in history.

I was hired to build their social media and community sites and tools (not alone, of course). In Overstock’s case, those “sites and tools” include blogs, forums, video sharing (a la YouTube), etc. I’ve had my fingers in basically all of these sites, but the big beast of late has been the forums.

Here’s the deal:

Overstock’s current forum system is built on phpBB2 and is very limited in it’s capability relative to the rest of the Overstock.com family of sites. The current project that I’m on has two primary objectives:

1. Migrate the Overstock forums to phpBB3
2. Integrate Overstock’s Unified Login system so that the forums authentication/authorization is based on enrollement in one of Overstock’s main sites (i.e. auctions, shopping, etc), rather than enrollment in Overstock’s forums.

Obviously the intent is to make the forums feel like a part of the Overstock.com family, rather than a third-party solution that has been crimped onto the corner of the Overstock sites.

So, why blog all this?

Simple: phpBB3′s code base is a freaking nightmare. So whether for my own future reference (never know when you’ll need it), or for some other poor geek that gets saddled with similar objectives, I figured I would document how I finally succeeded in making it all work. Obviously I’ll leave the specifics of Overstock’s unified login system out–that’s just ethics and good sense. But given that phpBB3 is open source (probably would be better referred to as “Obfuscated Source“) I am perfectly justified in divulging all I now know about its inner workings. Sadly, even after many epic battles, I still don’t understand much of how it all works. What’s worse, I think, is that I don’t understand why phpBB3 is built the way it is. I figure it’s for one of two reasons:

1. They have built their latest release on a horrible legacy code base with only enough modification to feel justified in calling it a new version. I could be wrong: they might have built it fresh from the ground up. If so, I’m even more disgusted by the phpBB development team, because that code is a mess. I find this hard to swallow, however, because the mere presence and pervasiveness of global variables throughout the code base tells me that the system design was lacking in foresight and flexibility, and was rather built to accomodate features that are making an unaltered journey from phpBB2 to phpBB3.
2. They have intentionally built on a hellishly cryptic architecture to keep it as proprietary as “open source” software is legally allowed to be. It’s certain that, once they’ve seen the code, developers and companies are going to think twice about building “their own” bulletin board systems by extending and modifying phpBB. The smart money would likely decide that they need to either stick with what phpBB offers (few modifications), or build their own system from scratch.

Of course, all of this could be completely unfounded. It could very well be that the code isn’t that hard to follow…once you know what to look for and where to look. But therein lies the biggest beast of all: trying to follow any logical flow through the phpBB3 code. In my exprience (and taking a page from the Simon Cowell Method of Introducing Similes), it was “quite like” tracking the movements of quantum particles at distances increasingly closer to the Planck Length. Every once in a while, with some heavy calculations and a fair amount of predictive luck, your observation might cross paths with the object you’re trying to observe; but mostly, the object just bounces seemingly instantaneously from one position to another without warning or trace.

At this point, I have to give props to the people at Zend. Were it not for Zend Studio (5.5) and their Zend Server Platform’s remote debugging capability, I’m certain I would have tossed in the towel weeks ago after determining that it was impossible to follow through the code manually. I was successful, ultimately, entirely thanks to the ability to step over and into specific parts of the code along the execution path; something that’s impossible to do in PHP without this Platform tool, as far as I know.

Let the battle begin.

I’m basically the poster child for Callaway. With the exception of my putter, my 5 wood and my hat, everything else I play is Callaway. It started about 4 years ago on my birthday. My wife–sweet and golf-tolerant angel that she is–bought me (at my request) Callaway’s then brand new ERC Fusion driver. It was by no means an impulse buy. I did my homework…which basically means I hit every driver in the valley to try to find the one that felt right.

After much research and countless balls hit in simulators, I narrowed it down to the Callaway, the Nike 1, and a Titleist (model I’ve forgotten). A little more effort eliminated the Titleist, and finally, when I managed to hit about 10 sim shots in a row with the Callaway that were dead straight and long (well…as much as such results can be trusted in a simulator), the Callaway won. I have never regretted it and thus began my commitment to Callaway golf gear.

I’ve gotten to a point now where I really don’t even need to test or compare anymore. I know Callaway’s gear will suit my game before I even swing the clubs, as was proven when my wife (isn’t she the best!?) bought me a full set of X-18 irons for Christmas two years ago. I didn’t even have to compare them to others, and by spring, I knew it was the right choice. I dropped about 5 strokes off my average that summer.

Next came my 56 degree X-Tour Chrome wedge, and finally, after a lot of credit card transactions on the Blue card, I earned enough points to get myself a 52 degree X-Tour Vintage wedge…excellent pieces of hardware that chipped another 3 or 4 strokes off my game last summer (pun intended, of course).

The final addition was the perfect ball. After playing many, and liking many, I finally found my magic with Callaway’s HX Tour and HX Tour 56 balls. They don’t last very long in my game; I tend to scratch them up pretty good with my wedges. But they have given me what I never have had in the previous 6 years or so of playing golf: spin control. They’re soft enough that even a hack like me with no real spin control can get the kind of spin that will stop the ball on the hardest of greens (as much as can be expected). Distance has never been a problem for me–I’m a pretty big dude–so for me, feel is key. Connecting with HX golf balls feels amazing, which is to say it doesn’t really feel at all. If I swing right, I can’t even feel the ball. That’s how I like it. Shave another couple of strokes.

So in a few short years I’ve gone from a complete hack who didn’t even dare register a handicap, to a proud 8.

I guess I can’t say for sure that the equipment is entirely responsible for my improvements; however, I’ve come to believe that there is one single element in golf that makes a greater difference in lowering your score than any other. It’s not the elusive repeatable inside-out swing, it’s not the putt stroke, it’s not the short game or the drive. It’s simply confidence. If you can strike the ball with confidence, no matter what shot you’re trying to make, your chances of success increase by orders of magnitude. For me, the feel of Callaway clubs gives me that confidence to know that if I do my job, the clubs will do the rest.

Now…where’s my sponsoship, Callaway!?

The births of stars, the organization of galaxies, the expansion of the universe through space-time…all in my own head!