posted under category: General on November 24, 2020 by Nathan
Every year, thousands of FIRST Lego League (FLL) teams work on an innovation project of some type. This is in addition to the actual Lego-based robotics competition where they solve a number of robotics-based challenges. The competitions can be a lot of fun, definitely full of energy! The annual challenge (not the robot part) is always something new. This year it was creating a solution to make people more active.
This year, my two younger kids’ FIRST Lego League team had an idea about building a scavenger hunt in a nearby park. They came to me with the idea to build a mobile app, and talked about all these bells and whistles. Like any good engineer, I talked them down to something more realistic and achievable. Plus something that’s in my wheelhouse.
Our joint solution was to build a mobile-friendly web site. This gets away from publishing to multiple app stores, paying a developer fee on one of them in particular, and working around bugs from multiple codebases across the different platforms. Of course now that the web app is built, it would be possible to port it through something like Phonegap or NativeScript, but I think the web app will be just fine.
So let’s talk about the scavenger hunt! This team created a list of things to find along the walking paths, took photos of them, and came up with the application’s rules:
- There are 2 different scavenger hunts
- Show as many as 10 items at a time
- Each item has a photo that can zoom in
- Each item has a checkbox
- Once they’re all checked, you win - celebrate with fireworks
Items to find are things like certain kinds of trees, categories of animals (reptile, bird), or park locations like the gazebo or the lake. The team also wanted to make sure we had links to find more wildlife info.
After the app is published somewhere, we will put a sign up with a QR code that will link directly to the web app.
On our first work night, we set up VSCode LiveShare so that we could all work on the code together at the same time. This was a huge pain to set up, but was an incredible collaborative tool once it was all working.
On the last team night before the QR code sign was printed up, I integrated our Git repo with Netlify - we were up and running in our forever-home within two minutes!
Some of this technology really is just getting so much better. From the libraries I easily integrated, to the free hosting, it was a lot of fun to set up. Maybe the best part though was doing a project like this with my own kids. I loved it, and I hope we can do more things like this in the future.
So without further ado, Scavenger Stroll for Wannamaker Park in North Charleston, SC:
I hope you enjoy it, and I hope it inspires you to do something great.
posted under category: General on November 23, 2020 by Nathan
Let me start by saying that we home school. And not just this year.
Initially it was out of a desire to keep our first kid ahead of the learning curve. This smart one was reading before kindergarten. Hey it worked - she graduated a year early and is making her way through college.
Something unexpected that came from this screwball 2020 year was the opportunity to teach a class at a home school co-op. This is essentially a one-day-a-week school experience that teaches those subjects that parents don’t want to do at home. We unashamedly use this for English classes, among a few other things. Families can pick up a class here or there, or build their entire curriculum out of it. There’s nothing home schoolers cherish more than the freedom to make educational choices for themselves, so this works for a lot of people.
We noticed that most of the teachers here fall into a couple small categories: parents of the students, and retired teachers. My wife has a particular interest in personal finance, with the real goal of making a generation of young people who understand things like credit scores and compound interest. So with all the qualifications being met, she’s been teaching her favorite things to high school students.
This year I found some spare time somehow (anyone else with me on that?). My wife put the bug in my ear about teaching a coding class. I really wrestled with what that even meant, and what students could benefit from it. When I heard our last CS teacher moved away over the summer, I knew we had a gap - somebody had to teach a computing credit. So with all the qualifications being met… why not me?
Also, more importantly, how?
I want to teach real languages and programming skills. I want to teach students to actually code, to understand what the computer is doing, and to see things from the inside. All of the curriculum I could find for high school seemed to be based on Scratch or Legos. It’s not that there’s anything wrong with that, just that there aren’t a lot of professional jobs for dragging-and-dropping Scratch blocks.
Maybe I’m a rare gem or something, but I started coding web sites in high school, then graduated and started getting paid to do it that same week. We all know the bar is a lot higher now with front end development being nearly impossible to break into without a good mentor and who-knows-how-many years of practice, but I like to think it’s still possible to start a career coding web sites.
To be fair, I see some of the folly in my ways. There is a challenge with some kids when programming moves from conceptually understanding what to do, to typing the code out on a keyboard. Furthermore, curly braces and parentheses can get messed up in ways you don’t want to see. Maybe Scratch would have been smart. Maybe Python would have been wiser.
Class started in the fall. 17 students showed up! This is my first class ever, I’m pretty excited! Also nervous. Stomach ache nervous. Stomach aches aren’t a sign of COVID however so forget that! This is just like a conference talk or anything I’ve done before, I’m going to knock it out of the park just based on my vague charm and the meme humor in my slideshow.
Yes, there’s a slideshow! It’s a Google Sheets slide deck. I picked one bold presentation theme for the year and have stuck to it. We’re actually fully on the Google stack of education. I have a Gmail account through the school, a shared Drive, and we are suffering through the underdeveloped early years of Google Classroom, which is still better than most of the alternatives.
One of my students was staying home, hoping to skip past a possible first-wave of COVID-19 as soon as school started, so unless I want to video record the whole thing and post it for the class (hint: I don’t), I’ll need to have some way to transfer the knowledge across the internet so when he finally attends physically, he won’t need to catch up. Every week I post my presentation in Classroom. At first just for him, but then I realized that it’s a great tool for the students to review our lesson, check on what the homework was, and read my possibly embarrassing speaker notes. I love transparency when it comes to learning.
I told the kids that everything that happens inside a computer is knowable - something you can learn. It seems like it’s a sealed black box, but that’s only because you don’t know how to open it yet.
So we kicked it off. Leson 1, install VSCode and Node so you can run .js files. Lesson 2, hello world! Then onward and upward from there. We’re only 13 weeks in and that’s the end of the first semester. That’s right, I’m writing this in retrospect of my first half-year! How did it go?
There can be a mental hang-up when first trying to put code into text, as I said earlier. For some of the students, this was, and still is, a big mental block.
The different braces and brackets are confusing. Why do we use curly braces on if statements and on loops and on functions and on objects too? This is where I began doubting JS and reconsidering Python for next year. At least square brackets usually means we are talking about arrays, and parentheses usually means it has to do with functions.
A student dropped out! He was probably the youngest one in the class, 14. I don’t know if it’s a brain maturity thing that grants the ability to write software, but I think that could contribute to it. Honestly, with my ~3 week introduction, he could probably pick it up and ace it in 2 years. After that, I started pre-presenting the presentations to Elly, my 11 year old. She seemed to get it and helped me smooth over the rougher stuff for the next few weeks. She also wasn’t great at the technical parts of coding, but she understood the concepts. Elly is smart!
Typing speed Something that popped up right away was the fact that most of the class typed under 30 WPM, and we had a few hunt-n-peck typists. I started assigning an amount of typing every week, with the goal of getting at least 60 WPM with proper touch-typing form.
Editing text is a skill that nobody knows without being told. This is one that I remember learning initially, and I still learn new tricks pretty frequently. We had to talk about text navigation, shortcut keys, saving often, and I even had everyone install Prettier to clean up what they’ve written.
Creating files was an actual challenge. There are precisely 44 ways to do it with VSCode (I didn’t count, I’m sure it’s over 6), and I don’t like to force people to do it “my way.” Honestly i don’t even prefer a specific way myself.
The JS Debugger isn’t good enough. There are errors that don’t get shown correctly, and some random outputs that are unexpected, like “Undefined” output randomly and rendering of empty blobs when using
console.table. Students are easily confused about that. Then they as me, and I’m easily confused about it.
I assigned FizzBuzz, which is maybe a little sadistic after just a few weeks of classes.
One student nearly finished FizzBuzz before class ended. Everybody else took all week. A couple of them never even turned it in.
We made rock, paper, scissors to practice our skills. That was pretty fun.
Our big project was a card game. Make a card, make a deck, deal a hand to at least 2 players, then declare a winner using whatever rules you want. I recommended high-card, then add poker rules for bonus points. My star student did the easy stuff, then built a blackjack game. Everybody got a passing grade, at least.
I explained functions twice. Maybe three times by the end. It’s a complex solution to a complex problem.
My son wants static typing! He asked me about dealing a hand of cards, and how function parameters work. He noticed it was difficult to tell what the argument in the function was supposed to be. I’m pretty proud of him. Without knowing it, he was asking for a stronger type system.
Throwing errors. The class thought the verbiage was funny. Throwing and catching. We didn’t go over it except in passing, but the terminology is somewhat memorable.
That’s it for now. I have a lot more but can’t think of anything right now. Feel free to ask questions!
posted under category: Software Quality on February 13, 2020 by Nathan
Maybe you don’t remember, so let me remind us all how hard programming is.
I have four children. Believe me when I say that I have been trying to get them to program from a very young age, and so far it’s not all roses and candy treats. Usually there is an amount of familiarization that a flourishing person needs to have with a subject in order to comprehend it fully. Learning how to read, for example, takes sounding out letters, followed by dick-and-jane books, followed by frustration, some time off (usually months), and then retrying, usually with a higher amount of success. Learning to read is a long process and doing it well involves creating many new brain pathways.
Imagine trying to teach your parents to program. With only a few exceptions, there’s a very high chance they simply won’t be able to get it.
This is something I wrestle with, and recently I’ve come to the conclusion that the standard human brain can’t easily grasp the concept of templating. Instead of building something, we programmers are building something for someone else to build something. The more meta up that chain our brains can go, the better a programmer we tend to be. Problems are always solved on a higher level than they were created.
Let’s say that we learned programming, we make our first application, we send it out for our users to try it, and what happens? It doesn’t work! We thought of every conceivable possibility. The problem was that we didn’t conceive of enough possibilities.
“Software is not governed by Moore’s Law; more like Murphy’s Law”
– Douglas Crockford
Even when we find a good solution, that doesn’t mean we’ve reached programming bliss.
“Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.”
– Jamie Zawinski
Sometimes progress is very hard and it can literally take years to get past our hurdles. My friend Ben wrote this a number of years ago:
“Every time I think I am making progress, I come to realize that I only have more confusion over the ‘right’ way to implement something”
– Ben Nadel
But don’t forget, programming is very difficult. It can be near-infinitely complex. Think about this for a couple minutes:
“Computer programs are the most complex things that humans make. Programs are made up of a huge number of parts, expressed as functions, statements, and expressions that are arranged in sequences that must be virtually free of error. The runtime behavior has little resemblance to the program that implements it. Software is usually expected to be modified over the course of its productive life. The process of converting one correct program into a different correct program is extremely challenging.”
– Douglas Crockford
If you’re programming, you are doing something extremely intricate. You are valuable because most people cannot do what you are doing. But wait! Your value isn’t only in programming, you are especially valuable because of your ability to put business processes to computers, and to explain these computer systems back to the business.
posted under category: Software Quality on February 7, 2020 by Nathan
One of the oldest and rawest forms of software measurement is the inimitable count of the number of lines of code. Let’s talk about that.
“Measuring programming progress by lines of code is like measuring aircraft building progress by weight.”
I work at an airplane manufacturer. We actually know the final delivery weight of an airplane, even adjusted for paint and seating arrangements, and in airplane manufacturing, knowing the current and final weight could help. On the other hand, a work of software is done when all the features are complete and the bugs are worked out, or better yet, when we know it will make or save enough money to start selling or using.
Lines of code differ drastically between systems. Choice of programming language is one of the primary reasons for a drastic change in the lines of code. One lower level language may allow us to implement a solution in 100 lines of code, while a higher level one has it as a built-in-function - one line. That doesn’t mean either language is worse, just different. Typically we see higher level languages develop applications much faster and debug easier thanks to the smaller codebases, but there is a tradeoff in performance and resource usage.
Another distinguishing factor is what function the application is expected to perform. It’s hard to compare a shopping cart system to a manufacturing system, they have completely different uses and thus a line count to compare them does essentially nothing. Perhaps one application is more straightforward and requires less code, even if it produces more in some kind of measurable output.
Then there’s the people problem. A developer on one system may have clever tricks that reduce the line count. This could make the codebase equally better or worse, generally depending on the cleverness of the solution, and thus the cleverness of the programmer. We often think we are smart. 90% of programmers believe they are above average, thanks to the cognitive bias called illusory superiority.
This then leads us to the yin-yang problem. More features need more lines of code, however there is a tax because more lines of code become harder for the software writer to comprehend and modify successfully. This is the exact nature of software complexity, and leads to the introduction of bugs, the creation of testing frameworks, the discussion of software quality, and the entire software industry. Remember that no lines of code means no bugs, no rework, no expensive developers, and no security hacks.
Next, we have to consider how the number of lines are calculated. Does that count include 3rd party libraries? Does it include blank lines and whitespace? Does it include comments? We can make a strong case both that all of these should or should not be included.
Lines of code does tell one one important thing however. It tells us how many lines of code are in the application’s codebase. It can give us a general order of magnitude for what we will find. It can give us the feeling of largeness, especially when we compare file-by-file (though typically a byte count would suffice). That’s it. Number of lines of code can tell us how many lines of code there are. That is all.
posted under category: Software Quality on February 1, 2020 by Nathan
Why do we want to measure software?
Software measurement is what we software developers do instinctively when we start a new job, join a new project, or enter a new team. It’s critical for us to understand the scope of a product so that we can learn our boundaries and understand our place. Understanding software is one of our best skills - the others being the ability to explain it, and the ability to write it (okay, there may be more to it). Our fight against the chaos of the unknown requires that we split up what we can observe into categories that we can understand.
Many times I’ve been give a legacy codebase without a proper introduction. That’s the worst feeling, and was my motivation for writing a big post on how much documentation you should have. If we could somehow bottle up this knowledge and pass it on to our colleagues, it could be very beneficial to future maintainers, helping to lower the long tail of operational cost on a product, and maybe even stave off the inevitable retirement of an app and the cost of rewriting it for a long time. And that’s our goal - it’s to remove cost, not to add it. It’s to understand the world and to be able to explain it.
Good measurements of software can bring a valuable understanding to us and to the people paying our bills. Is the software tiny and easy? That’s good to know! Is the software humongous and complex? That’s good to know! Is the software terrible and in dire need of help? That’s good to know!
How do we measure software?
Answer that and you’re likely to get a grant. The truth is, software isn’t specifically measurable, for the same way we can’t make software perfect. Sure, at a certain small size we can verify that it works as expected, but, you know, can it read mail yet?
Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.
That’s tongue in cheek. These days we could change Zawinski’s Law to interacting with social media or running as a native mobile app.
Our trouble with measuring software is that measuring by any single logical metric is a sure way to miss the mark altogether. Software measurement has to be on a different level, in fact, many different levels. Measuring software can only be done when we take in a system holistically. That means understanding it on a much broader level.
As soon as we measure the size on disk, somebody’s in there changing things. As soon as we measure the memory footprint, someone logs in, adding data. As soon as we measure the performance, another proces ties up the CPU and throws off our measurements. Software is soft, and changes too frequently to make accurate measurements that matter. Software is fluid, moving from disk to RAM, and transmitting across networks. It doesn’t take much of a program to utilize all of the hardware given it. It seems that simple metrics just won’t be enough to measure software. Measuring software by its metrics can give a picture of the whole, though, and there’s value in that.
When thinking holistically, one of the aspects that would need to be measured is the quality of a work. We need to measure the quality to complete this picture.
How can we measure software quality?
There are more intangible ways of measuring software, such as number of customer requirements, size or activity of the product’s backlog, and number of users. In particular, we can concentrate on the number of logged defects, if they have been logged at all. None of these are a holistic measurement of the quality of a piece of software, but they do help to paint the broad picture.
I believe that quality is a collection of mostly-intangible aspects. This means quality cannot be measured with a number or a picture. I think maybe a dozen pictures and user interviews and developer deep dives could help complete a mental model of the quality of an application.
There are quality metrics, such as the amount of code coverage (amount of code that is covered in unt tests). That’s a potential measurement of quality, but it could still be a terribly built app with great unit tests. Cyclomatic complexity is one of my favorites, having built a McCabe Indexing tool of my own a few years back. It tells you how complex your CFML files and functions are, and can show line-by-line where your most complex areas are. Another one I’ve looked at is function point analysis, which can map the central parts of an application by usage.
Once a software analyst understands the system holistically, they should be able to express that holistic idea to others. Of course this is a breakdown of its own type, and relies on the communication skills of the analyst.
What are the aspects of software quality?
I’ve compiled this list that I’ll go over at a later date, but feel free to suggest more. Quality is:
- Client Portability
- Server Portability
posted under category: Software Quality on January 30, 2020 by Nathan
The official software crisis was about computing power versus software power in the 1960’s. Hardware was outpacing software, which led to the need for higher level languages, better development methodologies, and more workers to write programs. Ask anyone in their later years who had been a programmer for their career and you may learn more than an earful about these problems. Something ongoing from the old software crisis were some of the signs and symptoms that we see today:
- Software is over budget
- Software is delivered late
- Software is low quality
- Programs run slowly
- Results don’t meet requirements
- Code is unmanageable
- Sometimes applications are never completed, vaporware
We see a good amount of that today. Personally, I see it all over enterprise software, and frequently in outsourced programs – that’s another topic for another day. I also see it in inexperienced programmers, especially when they are allowed to work alone, and sometimes in development done outside of normal software companies and departments. For real though, sometimes we even see it in more experienced looking teams as well.
The struggle is real. A lot of this software crisis is still in effect. It just doesn’t act like it did back before I was born.
Better software lifecycle processes typically help with late, over-budget software and with missed requirements. I personally have seen how Agile was a giant victory over the 6-month deployment cycle in years before. Tools like CI/CD enablers can put working code in front of our users even faster. Unfortunately, these processes don’t remove the problem of low-quality code, though they help mitigate many of the hurdles.
Isn’t that just software life though? We struggle through it every day in order to make the world a better place. That’s the programming life. That’s what we get paid to do. Maybe it’s kind of the dark side, sure, but that’s where we live. What do you think? Is the software crisis still a term worth using today?
posted under category: Software Quality on January 26, 2020 by Nathan
I really enjoy a lot of the content on Y Combinator’s Hacker News. One recent article that came up was Why do we fall into the rewrite trap?, which was a great look into the anti-pattern of software rewrites instead of thoughtful refactoring.
The comments tend to be as good as the articles, for example, one commentor mentioned Theseus’ Gambit - a philosophical engineering question about replacing every plank on a ship, at what point is it a new ship? This is a great question when it comes to software rewrites. Isn’t a refactor somehow creating a new work? Or is it only a rewrite if you start over?
So many comments agreed with the author and added their own notes. One wrote “rewriting a codebase is a lose-lose proposition: I’ll never suggest or recommend it again.” Others mentioned the need to rewrite based on technologies, like old Flash applications.
Not long after that, #TeamRefactor got a victory on a desktop app - it was a bigger team and while the code was bad - I mean like, a complete misunderstanding of OOP - we decided to refactor a lot of it instead of starting over. It was a somewhat political decision because we didn’t want to make the previous dev team angry. Unfortunately, they were equally offended that we were planning a refactor. Some drama took place. The refactor effort took a long time. We finished it eventually, but we ended up with an app that we still didn’t like (it was better, and many of the initial underlying problems were fixed, but…), our friends didn’t like us anymore, and a lot of time was wasted - far more than we expected. I would vote that refactoring this app was an overall failure.
A little later I was given charge of an IT app that was built only 6 months earlier and still in active development. Its foundation was in ASP.NET WebForms and VB.NET. After I gagged a tiny bit, I swallowed my pride and tried to hunker down and start on improvements. Backing up, our department has been a C# MVC group for some years, and when we follow one path, we try to make the rest of our choices as vanilla, gravy, plain, and standard as possible. This helps with ramping up new employees. The group that started this app chose .NET but then went with every other different choice, probably just based on familiarity. I took a task off the backlog for creating a mobile-friendly version, which turned into a reimplementation of the entire app, running much faster, was natively responsive, perfectly printable, and ended up being the spearhead of the larger vision for version 2: a complete rewrite. I did it with tact, not offending any parties, announcing my successes and advertising my work while helping the product team make the right choices on the vision. The end result was a much better product that fell closer in line with department goals. Another point for #TeamRewrite!
Not long after, I was handed a C# MVC + React app. The current steward was about to leave the company, so I had a transition week and then it was mine. The app was fine, there was nothing really wrong with it. On day one of independence I had about 30 commits to fix a bunch of things that I thought needed changing but didn’t want to do it and stomp on my former co-worker’s code and feelings. Very quickly, a number of things started working better, I caught up with all the changed from the data team that worked out of our Seattle office, I refactored some things that probably needed it, created my own backlog, and pretty much got bored about 3 weeks in. This is when I started to rewrite the UI in Vue.js. There were some requirements coming down that would have me using Redux, and I do not want to use Redux again, also I couldn’t figure out React-Router’s chunk splitting (I guess it depends on
React.lazy()), and I don’t like the React runtime’s bloated file size. On the other hand, I’m a huge fan of Vue, which is smaller, faster and I just like it more. I’m opinionated! I was pretty sure I could keep up with the React app changes and create the Vue version (I could), chalk up the UI change from React-Semantic-UI to Vuetify (Material) as just being a visual update (I did), and get it all done in 4 weeks (I didn’t). It took me 6 weeks. I floated it with the data team and they were positive on it, so I pushed it to the users… they barely noticed. After a while I got some comments about how much they liked the changes. The result was an application that ran a little faster, downloaded quicker, was more maintainable, could easily support all the new requirements, and I just plain liked it more. #TeamRewrite for the win. Was it worthwhile? Maybe, until I consider the pain I would have to go through for implementing Redux, then it was definitely worth it!
So what have we learned?
Maybe that Nathan just likes to rewrite applications for fun. Am I an anomaly if I keep having a lot of success with it? I work on internal enterprise systems; I wonder of the applications I keep inheriting are just really, terribly bad.
For the record, I’ve worked on plenty of systems that I did not suggest rewriting. I have one app that I inherited and have kept alive for twelve years now. It’s been through some major refactors, but it’s been evolutionary the whole time. Of course now it’s time for a rewrite - haha! It’s ColdFusion after all, and we don’t do that anymore. How did this application avoid Nathan’s swift justice against poorly coded enterprise apps? Well it was very well structured from the start - strong MVC roots, layered architecture, it was built on a firm foundation. Maybe #TeamRefactor should get a few more points.
While refactoring is typically less expensive, I have a list of reasons that will ping that rewrite bell in my head.
- When the existing application is very obviously, truly a disaster
- The UI looks like it’s 20+ years old
- When the existing technology platform is inadequate
- When upcoming changes require significant rework
I remember reading - though I can’t recall where - a hard figure. It was something to the extent of: when more than 60% of an application or component needs to be refactored, that application or component should probably be rewritten.
I really don’t prefer to rewrite, and neither should anyone, however, I believe that rewrites are much more possible than we are led to believe.