But that’s not enough.

The existential threat posed by SOPA to the very foundations of the Internet as we understand it can not be ignored and GoDaddy’s support of the bill — even now that that support has been withdrawn — was more than enough to send me and thousands of other technology professional packing.

The lesson of GoDaddy is this: there is no crime more abhorrent than to betray the very livelihoods of your customers. If the position of your company seeks to jepordize not just my ability to purchase your service but my ability to feed my family than no amount of discounts, stellar service, or witty ads will convince me to stay.

Of course, GoDaddy is a drop in the bucket; the list of SOPA supporters is long indeed and honestly very few of them will face a boycott because of that support. The internet is a mercurial and savage disciplinarian, however, and while few SOPA supporters will face the kind of reaction GoDaddy endured, many will no doubt fear it. The question is, will that fear translate into a change in policy?

Then again, there are clear cut cases where marketing is done well and its done badly. Apple knows how to market. Google does not.

Let me state, from the outset, that I’m an Android fan. I got an OG Droid (that’s the original Droid for those of you who don’t frequent Android news forums) back in 2009 and I’ve been hugely happy with it. No, it’s not as snappy or as polished as an iPhone, but it can do a lot of stuff that an iPhone can’t and I can tinker with it, which appeals to my more dwarfish (in the Tolkienian sense) sensibilities.

So one can imagine how pleased I was to learn that not only is Google going to release its next generation phone very shortly and that it will be running the latest and greatest version of Android to boot, but that said release lines up fairly nicely with my eligibility for a phone upgrade.

Except… that was two months ago.

Since then, Google has gone back and forth on the release date for their phone. They’ve hosted a lack-luster demo of the technology in Hong Kong (of all places) which ended up being an exercise in watching an executive awkwardly stumble through features with neither enthusiasm nor any real finesse. They’ve even confused the issue by refusing to stick with a consistent naming and branding strategy. I honestly don’t know if the phone — which is apparently due out later this month — will be the Galaxy Nexus or the Nexus Prime.

Imagine, if you will, if Apple had run this product launch. For starters, there would be no branding confusion. Google’s original Nexus isn’t a going concern anymore. They’d call the new phone the “Nexus S” or something like that (which everyone would immediately mentally shorten to “Nexus”) and have done with it. They’d have gathered a bunch of media, fanboys, and bloggers into a huge auditorium in California somewhere and done the launch live in the AM hours, thereby ensuring that everyone in their major target markets (the US and Europe) can watch it live either at work or at home. Finally, they’d have committed, on screen and in huge easy to read type, during the presentation to both a price-point (with contract) and a release date. Preorder screens would have gone up in the middle of the presentation and by close of business that day folks who wanted the new phone would have to do nothing but wait for the drop date with an optional “wait in the overnight line for the phone because you’re crazy” event for a midnight release.

This isn’t difficult stuff. Apple products have their shortcomings but if they’ve done one thing it is establish to almost formulaic accuracy, the correct way to launch a piece of consumer electronics. This requires no great Madison Avenue agencies, no fantastically expensive MBA laden consultancies, just a willingness to emulate success. Instead, Google as opted for a strategy which has confounded and frustrated its most devoted customers at every turn and a lot of people have chosen to go with a competing device rather than try to guess when the Galaxy-Nexus-Prime will drop.

Android has made some impressive gains on the smartphone OS market these last few years but Google performance in this launch forces me to conclude that this isn’t because of, but rather in spite of, the company’s understanding of the consumer electronics market.

If you’re not familiar with Group_Concat, consider a basic role based security system.  You have numerous users, each of which has has one or more roles.  It is easy enough to get back a list of user/role pairs, easy enough to get back a list of roles with one user per role, and easy enough to get back a list of users with one role per user; that’s all basic join syntax.

But what if you want a list of roles — one row per role — and want to know every user that has that role?

That’s what Group_Concat is for.  Bicker and argue all you like about why one might or might not need this functionality, there are just times when it’s nice to have.

Unfortunately, MS-SQL doesn’t really implement it… or it didn’t.  Recently I found a sort of hacked together implementation of Group_Concat in MS-SQL that actually works.  Since I’m developing in C# these days, the following implementation applies to a SQL 2008 database implementing ASP.net’s standard role based security.

SELECT
     RoleName,
     UserNames
FROM
     dbo.aspnet_Roles AS r
     CROSS APPLY (
          SELECT
               u.UserName + ','
          FROM
               dbo.aspnet_Users u
               INNER JOIN dbo.aspnet_UsersInRoles uir
                    ON u.UserId = uir.UserId
               INNER JOIN dbo.aspnet_Roles rr
                    ON rr.RoleId = uir.RoleId
          WHERE
               r.RoleId = uir.RoleId FOR XML PATH('')
     ) D(UserNames)
     GROUP BY
          RoleName,
          UserNames

The relevant parts of this are the XML cleverness and the CROSS APPLY, everything else is basic SQL.

The XML bit relies on MS-SQL’s XML support and is, bluntly, a hack. You can play with it independently with queries like this one

SELECT
     UserName + ''
FROM
     dbo.aspnet_Users
FOR XML PATH('')

Long story short, adding that FOR XML PATH(”) to the query results in instant concatenation of values. So now all that’s left is to properly relate this to the outer query. That’s where the CROSS APPLY comes in. CROSS APPLY allows you to specify what amounts to a derived table that can legally contain references to some outer information, in this case our outer query. That’s necessary to constrain our inner table so as to only get the users we are interested in. We can’t do it outside of the CROSS APPLY because, by then the concatenation has already occurred.

The others are spam. All of them.

This despite trying a range of various CAPTCHA solutions, filters, etc, these I’ve been unable to stem the tide. Indeed, those 1,288 are just the ones that got through. Another ~900 were identified as SPAM by WordPress and blocked automagically.

Bluntly, this is absurd and, as I think about it, the possibilty of designing around such issues is equally absurd.

Even Gmail’s spam filters are not perfect, despite the unthinkably large data store from which they can pull and the absurd resources that Google is prepared to throw at the problem.

Practically – at least to my eye – this problem can not be completely solved by computer science. The solution lies, instead, in the field of economics.

Spammers do what they do because the cost is effectively zero and the preceived value is greater than zero. With zero cost, even if the marginal utility – the preceived gain from each individual action – declines with each message or post, it never drops below the cost.

It is thus always profitable to spam.

The solution lies not in attempting to block spam, which is difficult, annoying, and labor intensive, but in reducing the preceived value of a spam message.

The “no follow” link convention implemented by Google helps — links so tagged (and most in blog comments are) are not given weight by Google’s search algorithms, thus eliminating the value of spamming for page-rank. Of course, that is not the only reason people spam.

Some are looking to sell a product – and view spam as free advertising. Others are looking to ensnare victims in some sort scheme – either financial or technological.

In both cases, spammers are hoping that humans, not computers, will follow their links. Some social networks like MSNBC’s Newsvine have taken to banning linking entirely – at least on recently registered accounts. Spammers then break up URLs or instruct users to search for certain phrases.

Ineffective? Sure, but again, with a marginal cost of zero, who cares?

Perhaps it is not possible to reduce the value of spam postings to zero without destroying the possibility of meaningful comment as well. If so then the only remaining option is to find a way to raise the cost of posting.

CAPTCHA was an attempt to do this, though one that obviously has met with limited success. Back in the 1990s the notion of a pay-to-send email system was floated with micro-transactions supporting the system and making bulk-spam mailing actually cost something.

That was 20 years ago, however, and there’s been little meaningful progress.

Sadly, I have no profound conclusion to offer you, dear reader, save this one. Markets work both ways and the simple fact is that while I can not make it economically prohibitive for spammers to post, they have made it economically prohibitive for me to maintain a comments section on this blog.

Effective immediately, therefore, commenting on Nephandus is disabled. In reality it has been for some time as the volume of new-post notifications from WordPress lead me to simply ignore them and instruct Gmail to automatically delete them several months ago.

The second issue is that user controls are in their own scope and don’t have immediate access to the rest of the page. That is both a blessing and a curse; good control design means that the controls should be fairly independent of each other but a more pragmatic developer will also note that user controls exist primarily so that numerous copies of the same thing can be easily created and that making entirely self contained controls leads to a lot of unnecessary duplication.

Fortunately there are workarounds for both of these issues.

Regardless of view state, a user control can pass information and therefore trigger events properly using the ASP framework by means of some elaborate JavaScript trickery. In this example, I am using JQuery and JavaScript to allow a click event inside a ViewStateless user control to set the value of a text box and click a button outside of the control, thereby triggering an event within the ASP framework.

$(JQueryObjectForMyUserControl).find(".clickableElement")
.unbind("click").click(function() {
        var clickHandler = $(this).parents(".OuterBound").find(".ClickHandler");
        //I'm replacing the underscores with dollar signs here because that is what ASP expects to find internally
        var argument = $(this).find("a").attr("id").replace(/_/g, "$");
        clickHandler.find(".ClickHandlerArgument").val(argument);
        clickHandler.find(".ClickHandlerLinkButton").click();
    });

Getting around the scope restriction requires a great deal less insanity. For example, my user controls have to be able to activate a modal overlay and dialog box. I could put the code for that inside each user control but that would be a lot of code bloat. Instead, I have the dialog in one place on my page and allow the controls to set its values and activate it.

Page page = this.Page;
ModalPopupExtender ModalPopupExtenderLockedDialog = (ModalPopupExtender)page.FindControl("ModalPopupExtenderLockedDialog");
Label LabelMessage = (Label)page.FindControl("LabelMessage");
ModalPopupExtenderLockedDialog.Show();
LabelMessage.Text = "Hello World!";

Getting to properties is only a little harder. If you need to get to the properties of your page you need only cast this.Page appropriately.

Not this kind of XML

VirPack has me working on an application that needs a user administered list of options for people to choose from. It’s a fairly simple thing that doesn’t require relational tables so I thought I’d toss it into XML.

Now, C# deals with XML really well. Almost everything, if you just ask nicely, will be quickly and easily packed off into XML the framework with just a few lines of code.

String path = Server.MapPath("~\\myfile.xml");
using (FileStream sourceFile = new FileStream(Server.MapPath(path), FileMode.Create, FileAccess.Write))
        {
            XmlSerializer serializer = new XmlSerializer(typeof(Foo));
            serializer.Serialize(sourceFile, bar);
        }
        bar = null;
        using (System.IO.FileStream sourceFile = new System.IO.FileStream(path, System.IO.FileMode.Open, System.IO.FileAccess.Read))
        {
            System.Xml.Serialization.XmlSerializer serializer = new System.Xml.Serialization.XmlSerializer(typeof(Foo));
            Foo bar= serializer.Deserialize(sourceFile) as Foo;
        }

As simple as that I’ve packed my class off to be serialized and then unserialized it. Three lines of code is hard to beat for XML serialization and deserialization. “Foo” is a user defined type, however, and my particular task being the population of a drop-down list, no such complexity was required. Yet, when I tried to feed a List of ListItems into the Serializer everything broke.

//This breaks
System.Xml.Serialization.XmlSerializer listSerializer = new System.Xml.Serialization.XmlSerializer(typeof(List));
//This too
System.Xml.Serialization.XmlSerializer typedListSerializer = new System.Xml.Serialization.XmlSerializer(typeof(List<ListItem>));

This has to do with how C# deals with collections. Serialization requires a root level node and a collection, despite having an appearance of such, has no root level information. It is a collection, not a collection bounded by something else and XML serialization requires that something else.

So C# has a hackish workaround. The ListItemContainer class is a simple type that contains a List of ListItems.

//This works
System.Xml.Serialization.XmlSerializer listSerializer = new System.Xml.Serialization.XmlSerializer(typeof(ListItemContainer));

Of course, not everything you’ll wish to serialize in a list will be a list item, but the fact that C# had to create it’s own little wrapper class should be a clue for developers. Getting a collection into XML requires a wrapper class; it’s as simple as that.

Would it have killed them to document that though?

Image Credit: PixelPlacebo via Flickr and Creative Commons

Unit testing is good; test driven development is better.  As Knuth once famously quipped “Beware of the above code. I have only proven it correct, not tested it.” There really is no substitution for good, solid testing.

Unfortunately, at least in C#, webpages don’t like to be unit tested.  I approach this post with an uncomfortable realization that I am about to lay out the issues and problems I’ve had while the solution I am presently using is far from satisfactory.

There is a right way and a wrong way to undertake a lot of unit testing.  If you want to learn about that, I recommend this post by Ryan Hagan as a great jumping off point.  My problem is not in writing good unit tests, however, so much as having a place to write them.

Axiomatic truth of Unit Testing: Your test code should exist in a different place than your production code.  This just makes good sense.  You don’t want to publish your tests every time you push code out the door and, since you want all of your published code to be tested you don’t want to have to write tests for your tests.  In C# the way one generally accomplishes this is the creation of a separate test project within the current solution.  That way the solution contains both your code and your tests.

Now generally, when you’re writing tests for C# this isn’t a big deal.  Since your project gets bundled up every time you build you can add a reference to the Test project that points at your main project; that way, the test harness knows what to make of all the custom types and whatnot that your various tests will have to evaluate.

This does not work for webpages.

When C# builds a webpage there’s no handy-dandy compiled thing at which to point your test code.  That means your test code has no idea what to make of any custom objects you’ve created and that makes any meaningful testing pretty much impossible.  Consider the following short example:

public void CreateTest()
        {
            string WidgetName = string.Empty;
            Widget expected = null;
            Widget actual;
            actual = WidgetFactory_Accessor.Create(WidgetName);
            Assert.AreNotEqual(expected, actual);
       }

In this example, “Widget” and “WidgetFactory” are both custom types.  C# has conveniently generated an accessor for my Factory but it has no reference to my main project and thus doesn’t know what a Widget is.  Consequently, this test won’t compile and I can’t run it.

There is a work-around solution I’ve found wherein I publish the Main Project and then create a reference from the Test Project to the published AppCode.dll file.  This works but requires that I publish the DLL every time I wish to re-evaluate my source code which makes the entire testing process prohibitively time consuming.

My suspicion at this point is that what I am trying to do is in fact neither possible nor recommended and that unit testing of a C# webpage should be accomplished by creating a sharp distinction between the interface and presentation layers, testing the presentation layer traditionally and then using interface testing suites like Watir.  This seems the sort of thing that should be verbosely stated somewhere if it is, indeed, the case.

Update

As per Ryan’s suggestion below and some other feedback I’ve gotten I am moving much of the logic code out of the project and creating a project-level distinction between the interface and the presentation layers of the application.  This should allow me to more sanely test the presentation layer and leverage Watin in to handle the interface testing.

It’s not the sort of separation recommended, but the architecture on this particular system is fairly dated and a full out re-factoring is out of the question (for now).

As it’s conspicuously missing from many other write-ups I’ve seen I’ll say it here.  You can not meaningfully test anything except the interface of a C# web project.  If you want to test your code, move it into a DLL project and allow your web project to call that DLL.  Ideally the Web project should be as thin as possible.

How do you user test this?

Most of the blogging I’ve done in the past has been political and thus I really haven’t had an opportunity to try to post much in the way of source code in a blog.  Once I got Nephandus up and running on WordPress, however, I thought I’d have a shot at it and thus posted a short article on my experiences with C#’s serialization quirks.

To illustrate a specific point I included a brief snippet of C# code which WordPress promptly turned into an illegible mess.  The web is a notoriously difficult place to display source code and thus I set off in search of a WordPress plug-in that would allow me to do so without too much thought.  Several days, a dozen plug-ins, and a string of curse words that would make a sailor blush with shame, I have a solution.

There is no shortage of syntax highlighting plug-ins available, but the support for their instantiation is practically non-existent.    Nephandus is running a number of plug-ins, none of which required much more than a few mouse-clicks to install and configure yet this particular task proved more difficult and involved than anything else I’ve done with WordPress.

The more I thought about this frustration the more I realized that this is an ongoing problem in the software development industry.

When developers buid a word processor or an email application they spend (or they should spend) a fair bit of time working out where users will intuitively look for certain basic functionality.  Tools, options, and commands should be where the user expects them to be; that saves time and cuts down on support costs.  We can test and evaluate that design by asking people who are totally unfamiliar with the product to find those sorts of functions and watching what they do.  It’s a system that works well because, familiar with the product or not, everyone has a basic grasp of certain tasks like writing or arithmetic.

But how do you do that kind of testing with an IDE?

The people who use IDEs are, pretty much by definition, super-users.  At the very least we can expect them to have a high level of familiarity, if not with the interface itself than with the thing the interface manipulates – code in a programming language they know.  These users have very specific ideas about where things ought to be which are based, not on intuition but where they were in some other IDE that they learned.  Because the population of knowledgeable coders is small and the number of IDEs they develop on is also small, it’s likely that they’ll bring some of their preconceived notions into user testing and that those notions will persist through the testing process.

Neophyte users aren’t terribly helpful because while they’re learning the interface they’re also learning the language and thus don’t know enough to evaluate the more powerful aspects of a product.

How then do you test the usability of a new interface when that interface’s intended users are a small, niche population often with preconceived, conflicting, and sometimes nearly religious notions of how the interface should look, feel, and act?

That isn’t a problem that WordPress plug-in authors really need to worry too much about – at least not if they’re giving their work away for free (beggars can’t be choosers) – but those of us who make specialty products for high-end niche professionals need to find a way to tackle the problem lest we find ourselves unable to change a 30 year old catastrophe of a design because it’s what everyone knows and expects.

People familiar with C# know that C# supports the inclusion of user defined controls called WebControls.  These are more or less very simple C# programs which can be man-handled by another bit of C# code.  They’re handy for making your code modular: you might design a web-control that takes and validates a credit card number, for example.

But WebControls are notoriously tricky beasts and over the course of the last few weeks I’ve come to understand that one of the reasons for this is that they don’t behave quite the way you might expect them to when they are serialized and deserialized.

Most C# serialization occurs behind the scenes in something called the ViewState.  For those of you who have no idea what that means, ViewState is C#’s rather clever way of persisting the things on a webpage so that it feels a little more like a desktop application.  The long and the short of its behavior is that things get serialized into the ViewState just before rendering and de-serialized back out of ViewState on the next request as part of the load step.

Most controls go into the ViewState… but WebControls don’t.  At least, they don’t go in there by default.  Presumably you can create custom ViewState serialization for them; I have not tried this.  Instead, I serialized my WebControls very simply into Session and coded that storage into a property.  My code looked something like this

private Widget InternalWidget
{
	get
	{
		Widget result;
		if (Session["Widget" + this.UniqueID] != null)
		{
			result = Session["Widget" + this.UniqueID] as Widget;
			if (result == null )
			{
				result = WidgetFactory.Create();
				result.Arguments = InternalActivePortlet.Arguments;
			}
		}
		return result;
	}
	set
	{
		Session["Widget" + this.UniqueID] = value;
	}
}

Now, as expected, none of my EventHandlers persisted when the WebControl came back out of its session serialization.  The object that comes back from serialization is an entirely new object; the old one has been destroyed.  As a result, the event handlers in the deserialized object point to null function pointers.  C# doesn’t throw an error when you call a null function pointer, however; it just goes on its merry way and does nothing.

But the function pointers – even though they now point to null objects – remain.  C# remembers a great deal more about the session-serialized object than one might think including – and this is the part that really threw me – the fact that it’s already been through the ASP parser.

I had expected, since my event handlers weren’t working when the WebControl came back from Session, that the events themselves were gone and that therefore the object returned from session was new — as if it was programatically created for the first time.  This is not the case.  When you programatically create a WebControl for the first time it’s ASP file is parsed and event handlers are set up based upon the code there.  When you deserialize a WebControl from session, however, C# remembers that it has already dealt with its ASP code and does not do it a second time.

Consequently, none of the event handlers defined in the ASP work either.

I expect that all of this seems rudimentary to more experienced C# developers, but to me it was quite a mystery and it took me some time to work out the above logic.  Hopefully this will aid some neophyte coder working through the same issues I had.

On a totally unrelated note: I need to find a decent code display plugin for WordPress…

Just remember to enable the “WordPress, Moveable Type, MetaWeblog and Blogger XML-RPC publishing protocols” in WordPress’ settings/writing section.