New Home

Hey folks, I’m running my blog in my own server now. Reach it in http://allanvital.com/blog/ .

See you there!

Advertisements

What is Mutation Testing?

Some time ago I worked in a project that was deeply concerned about percentage of code coverage. It is, indeed, a good thing, but something to be careful, because, even if you hit 100% of the lines, it still doesn’t mean your tests fully cover all possible mistakes that may happen. That occurs either because the logic can be, sometimes, hard to emulate or because the test was poorly written.

The second one is something that worries me. Tests are handy to give credit to code, in maintenance and in overrall system quality. In other words, tests assert the quality of your code (in the ideal world). But what do asserts the quality of tests?

That’s when mutation tests shine. Their objective is to make sure that, if something changes in your code, the tests will break. That’s incredible useful to show you where your tests are weak, and gives you confidence that all lines being executated in the code are there for a reason.

mutation

The full cycle of mutation

 

How does it happen

Roughly, what a mutation test suite does is: take your code, change something meaninfull (i.e create a mutant) and run the tests. They should break, and, if they don’t, it means that they are weak. Simple as that.

More specifically

The suite will get your code and make some interesting changes, like remove one line, change and operator (like “>” to “<”) or nullify an object. This should be enough to break some test, because, if it doesn’t happen, it means that your tests are hitting that line but not testing it. Putting it simple: if another developer makes that change, your tests should warn him about this.

Keep an eye for mutation test, it might be useful for your team.

Mysql – Removing the “Using filesort”

Hey there, deep specific post here.

There are many options out there, but MySQL is, by far, the most used free RDBMS. Nevertheless it is pretty common for us, developers, to be sloppy when it comes to optimize our queries. Of course, as I said here, premature optimization is somewhat dangerous, but I’m talking about these times when you really need to bring some query execution time down.

One of the greatest villains of MySQL query optimization is the dreaded “Using Filesort”. Trying to order something without an index can, in extreme cases, kill your query.

Trivia: Don’t be fooled by it’s name: “Using filesort” is, in it’s core, a quick sort (ignore the “file” in it’s name).

How does it happen

Almost any order query without indexes. My example will show a two fielded order by clause:

CREATE TABLE IF NOT EXISTS `nice_table` (
  `id` int(10) NOT NULL AUTO_INCREMENT,
  `field_1` varchar(255) NOT NULL,
  `field_2` varchar(255) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB  DEFAULT CHARSET=utf8;
EXPLAIN SELECT field_1, field_2
FROM nice_table
ORDER BY field_1, field_2

Running this query will give something like this:

ID SELECT_TYPE TABLE TYPE POSSIBLE_KEYS KEY KEY_LEN REF ROWS EXTRA
1 SIMPLE nice_table ALL (null) (null) (null) (null) 30 Using filesort

“Using filesort”, bad bad bad. Let’s solve this:

ALTER TABLE nice_table ADD INDEX idx(field_1, field_2);

With the index, it’ll order in the index itself, avoiding any post-query process to order the result set.

explain SELECT field_1, field_2
FROM nice_table
ORDER BY field_1, field_2;
ID SELECT_TYPE TABLE TYPE POSSIBLE_KEYS KEY KEY_LEN REF ROWS EXTRA
1 SIMPLE nice_table index (null) idx 1534 (null) 30 Using index

Now that’s better! Of course, in small tables it does not even make a difference in query time, but watch out in huge databases. This will cause a great performance boost.

Big hugs!

The Software team is not a “Black box”

When a Project Manager says “To finish this we need more resources” what I picture in my mind is this:

Just buy more resources!

Just buy more resources!

It really sucks to think of people as resources. It’s something that bugs me, and the pure association (developer = resource) brings some intrinsic crazy deductions like “developers are high quality typist”, “developers are just machines that turns requests to features” or, even worse, “we can just trade these resources (aka: developers) for new (and cheaper) ones”. Well, a piece of advise: the developer isn’t the tree, he is a lumberjack.

"Just fixing this damn bug!"

“Just fixing this damn bug!”

I think that this association (developer = resource) is the root of one of the main problems in medium and big companies: the “Black Box Efect”. It goes something like this: The products owners, managers and consultants go to a meeting to define some new feature. After some discussion they write down their conclusions and send it to be done by the development team. Then the developers create the features and everyone goes to the bar to congratulate each other.

Há, just kidding. The request will be detached from the current product architecture, it’ll demand much time and effort to be done and it will make everyone miserable: developers because the irreal feature and heads for the delays and problems. Futhermore, any new improvements will be harder to implement, just because of the existence of that “big ol’ fat feature”.

Of course, this is the worst case cenario, but it happens more often than you think.

These are the things that happen when you just include de development team in the final step (“Here, create this!”). The developers should be included from the beginning. This way they can help refine the request to be healthy added to the product. I think that the “old time developer”, that guy that couldn’t talk to people and just received requests and a bowl of food to code, should not be the reality now.

Real developers must be able to, consistently, explain to non technical folks what’s happening and why this request is not such a good idea. And not just the “Tech Leads”, but any developer in the team.

Damn you, Jboss 5!

Hey there, Allan here.

Quick post for legacy projects:

So, some time ago I was  allocated at a legacy project that runs on Jboss 5.1. I’m not a huge fan of the early Jboss versions (jboss 5 and 6… jboss 4 is not that bad), mainly because of my previous experiences with these application servers.

I was “checkouting” (damn, that’s an ugly word!) the code and decided to download the Jboss version needed. So I went to Jboss Downloads and download the JBoss AS 5.1.0. Downloaded, put in my Tools folder and hit run.sh:


Failed to boot JBoss:
    java.lang.IllegalStateException: Incompletely deployed:

    *** DEPLOYMENTS IN ERROR: Name -> Error
    StandardBindings -> java.lang.IllegalArgumentException: Wrong arguments. new for
     target java.lang.reflect.Constructor expected=[int] actual=[java.util.HashSet]

    *** DEPLOYMENTS MISSING DEPENDENCIES: Name -> Dependency{Required State:Actual S
    tate}
    ServiceBindingManager -> ServiceBindingStore{Installed:Instantiated}
    SystemPropertyBinder -> SystemPropertyBinder#1{Installed:Described}
    SystemPropertyBinder#1 -> ServiceBindingManager{Installed:Described}
    ServiceBindingStore -> StandardBindings{Installed:**ERROR**}

            at org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.internalVa
    lidate(AbstractKernelDeployer.java:290)
            at org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.validate(A
    bstractKernelDeployer.java:174)
            at org.jboss.bootstrap.microcontainer.ServerImpl.doStart(ServerImpl.java
    :148)
            at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:
    394)
            at org.jboss.Main.boot(Main.java:209)
            at org.jboss.Main$1.run(Main.java:547)
            at java.lang.Thread.run(Unknown Source)

It was a clean start, I didn’t even copied my .ear to the deployment folder. Just download and run and error 😦

The Solution

The Jboss folks were pretty lazy in that one: there is a known bug in Jboss 5, that makes it impossible to start it without, manually, correcting one of it’s xml files.

Inside the conf files, there is the profile.xml. Locate, inside it, the AttachmentStore bean and correct it, putting the class=java.io.File in the parameter tag, as shown bellow:

<bean name="AttachmentStore"  
class="org.jboss.system.server.profileservice.repository.AbstractAttachmentStore">
<constructor>
<parameter class="java.io.File">
<inject bean="BootstrapProfileFactory" property="attachmentStoreRoot" /></parameter></constructor>

Hope it helps 🙂

Get things done

Hey there, Allan here.

I think that one of the best parts of being a web software developer is the power you have over your application. In an offline product you need to care about older versions and compatibility issues, and it’s really challenging to find a good development pace.

(I know, if you build an API, you must take some of these in account too, but you still have full power over how you deal with it)

With this in mind you can find a good development speed and put things in production while they are still under heavy development. This way you can build an initial “user base” and see how your product/service behave in the real world. And this is what all we do is about.

Note that your user won’t know what he wants (even if he thinks he does), but analysing his data and his behaviour can (and should) influence your next steps. Therefore, put things in production should be you main goal. Focus on your “main functionality”, and when it’s working nice and easy, it’s production time.

I know that this sounds too agressive, or to crazy to actually work, but as some smarter folks stated, it works. Otherwise you will find yourself with a big frankenstein that tries to do everything everywhere.

Sounds familiar?

Sounds familiar?

Also, lookout for premature optimzation: sometimes we spend days trying to improve some routine without even knowing if it really needs to be optimized. And, in the end of the day, your customer doesn’t care if your code is extremely elegant, or just plain simple, as long it does what it needs to be done, and within the expected time.

Of course it’s no excuse for you to write shitty code and say “you have to be fast!”. In my experience, yes, you have to be fast, but fast in the long run. If you code bad, you might be able to put first in production, but will make maintenance harder… and, as we know, maintenance is where we will spend most of our time as software developers.

To be short, your job shoud be: analyze user behaviour/needs, define feature, code right, test right, put in production, repeat

The java.lang.NoSuchMethodError

Hey there, Allan here.

Quick post, for the begginers out there:

I think anyone who programmed for more than one month with a Java + Maven project already saw that error. The basic scenario is simple:

You got your project compiling and building nice and easy. You lay back, relax a bit, and write some tests, go to youtube, refactor some method, stare at your screen and admire your work.

Damn, Im awesome!

Damn, Im awesome!

After a while you decide to do a full test. You pack your code, throw that in your server, restart, test and voila…

java.lang.NoSuchMethodError

You just panic! How in the world is that possible?? You were sure of everything! And then you spend some time to figure it out what’s wrong.

The Answer

Dependency conflict. ALMOST aways that is the case: you added some dependency that is dependent to some other dependency from your project, with a different version. Let me illustrate:

Dependency ConflictIn this case, you depend on log4j-1.2.13, and when you add commons-logging-1.1, it will bring as a transitive dependency log4j-1.2.12. Bam! Error! Panic! Crying!

It’s not overly complicated to solve this, but, can be troublesome because there’s no easy way out here, you got to check your pom for dependency conflicts. Eclipse got a great maven plugin, that helps you out, but in case you want:

mvn dependency:tree

This will show you a complete tree of your dependencies. With that in hand, you can search for dependencies that caused your headache.

When you find the problem, the best way is to exclude the transitive dependency.

Hope it helps!

Using Apache as a reverse proxy to Jetty with SSL

Hey there, Allan here.

Since my last post was about ideas and discussions I decided to make this one more technical. So here is the cenario

I’ve got a maven project that runs with org.mortbay.jetty.maven-jetty-plugin. That way I’m able to run “mvn jetty:run” and got the system up and running in no time. Since it’s a “default” jetty installation, when I’m done with a new feature, I just pack the code and install in the jetty of the homologation enviroment.

The Problem

In unix systems you can only allocate ports bellow 1024 being a root. That means that, if I want to run as a normal user, when I set the Jetty plugin, I must choose some port over 1024 for both http and https (ports 80 and 443). The problem is that, if I use other ports than these, I must, manually, put the ports in the URL when navigating through the system. And that is WRONG.

So, as many of you may have guessed, when I’m working in a project with this setup, I put my jetty on port 8080 and configure an Apache Server to be a reverse proxy to my jetty. (since apache runs as a service, and, therefore, can run on 80)

<VirtualHost *:80>
ServerName myproject.com

LimitRequestFieldSize 65536

ProxyRequests Off
ProxyVia Block
ProxyPreserveHost Off
ProxyTimeout 10

ProxyPassReverse / http://127.0.0.1:8080/

RewriteEngine On

RewriteCond %{REQUEST_URI} ^(.(?!\.css|js|gif|png|jpg|ico))*$
RewriteRule /(.*) http://localhost:8080/$1 [P,L]
</VirtualHost>

And I was happy forever after until someone told me “hey, this one in https” and handled me a jks file to be the jetty certificate. So, I configured the jetty to the oncoming safe requests, on port 4430 and went to configure my apache.

Since it was a dev machine, and Apache don’t like jks, I created my own certificate for it, with openssl:

Creating certificate

openssl genrsa -des3 -out self-ssl.key 2048

This command will create a self-ssl.key file. As you may have guessed, this is my key.

Then I created my Certified Signing Request (CSR)

openssl req -new -key self-ssl.key -out self-ssl.csr

No news here, as it will be in self-ssl.csr.

Then, finally, my certificate

openssl x509 -req -days 365 -in self-ssl.csr -signkey self-ssl.key -out self-ssl.crt

(IMPORTANT: it will ask for a passphrase. Create one and remember to save it)

Using the certificate

With all these files in hand, I went to configure my virtual Host again:

<Virtualhost *:443>

ServerName myproject.com.br

SSLEngine on
SSLProxyEngine On
SSLProxyVerify none
SSLCertificateFile /path/to/my/crt/file/self-ssl.crt
SSLCertificateKeyFile /path/to/my/key/file/self-ssl.key
ProxyRequests Off
ProxyVia Block
ProxyTimeout 10
ProxyPass / http://127.0.0.1:4430/
ProxyPassReverse / http://127.0.0.1:4430/
ProxyPreserveHost on

RewriteEngine On

RewriteRule /(.*) http://127.0.0.1:4430/$1 [P,L]

</Virtualhost>

Then I restarted the apache, everything worked and I went for a beer…… há! Just kidding, the proxy stopped working.

Why?

Of course, as you see, I had two different certificates and they didn’t want to be friends. The logs accused problems with not being able to get a valid HandShake.

The quick and dirty solution

Point the proxy to the unsafe port!

<Virtualhost *:443>

ServerName myproject.com.br

SSLEngine on
SSLProxyEngine On
SSLProxyVerify none
SSLCertificateFile /path/to/my/crt/file/self-ssl.crt
SSLCertificateKeyFile /path/to/my/key/file/self-ssl.key
ProxyRequests Off
ProxyVia Block
ProxyTimeout 10
ProxyPass / http://127.0.0.1:8080/
ProxyPassReverse / http://127.0.0.1:8080/
ProxyPreserveHost on

RewriteEngine On

RewriteRule /(.*) http://127.0.0.1:8080/$1 [P,L]

</Virtualhost>

This way, your apache will validate it’s own certificate and proxy the requests to the jetty “http” port. Since it’s a dev machine, it’s not a security issue, and you don’t need to create “copies” from your real certificate. You will navigate trough a safe connection and can test all your system with no worries.

The Php Debate

I was thinking about my next post, when I ran into “The PHP Singularity” from Coding Horror.

I’m a big fan of Coding Horror as it is one of the best developers blog out there, and in this post he makes it clear that he thinks PHP as a “double-clawed hammer”:

I can’t even say what’s wrong with PHP, because – okay. Imagine you have uh, a toolbox. A set of tools. Looks okay, standard stuff in there.

You pull out a screwdriver, and you see it’s one of those weird tri-headed things. Okay, well, that’s not very useful to you, but you guess it comes in handy sometimes.

You pull out the hammer, but to your dismay, it has the claw part on both sides. Still serviceable though, I mean, you can hit nails with the middle of the head holding it sideways.

You pull out the pliers, but they don’t have those serrated surfaces; it’s flat and smooth. That’s less useful, but it still turns bolts well enough, so whatever.

And on you go. Everything in the box is kind of weird and quirky, but maybe not enough to make it completely worthless. And there’s no clear problem with the set as a whole; it still has all the tools.

Now imagine you meet millions of carpenters using this toolbox who tell you “well hey what’s the problem with these tools? They’re all I’ve ever used and they work fine!” And the carpenters show you the houses they’ve built, where every room is a pentagon and the roof is upside-down. And you knock on the front door and it just collapses inwards and they all yell at you for breaking their door.

That’s what’s wrong with PHP.

(go there and read the whole post)

Well, it’s his opinion, but I think that his opinion is wrong :), for several reasons… that’s the main idea in this post. So let’s go!

The Bad Parts

Slow

PHP is a scripted language. Trying to interpret and define and create and instantiate and blablabla during the request is slow.

Two systems running, one in PHP and one in Java, both doing the same thing: Java will be WAAAAAAAAY faster.

The nephew effect

Client: “So, how much it will cost?”

You: “It’ll cost X, and will take Y time.”

Client: “Go to hell! My nephew does this in half of the time and with X/10 cost!”

Sounds familiar? Yeah PHP suffers this A LOT, and it is nothing that happens with Python, for example. Almost anyone can code in PHP and you can imagine what will come out of this, wich brings us to…

The “legacy”

I can tell you three or four stories of “add one or two things in my aready running system, written in php”, all of them ending with me, laying in fetal position under the table and crying for my soul, but that’s just sad.

The big point is: since PHP is simple and quick, everybody can open a php file and add something, and by everybody, I mean EVERYBODY.

That leads up to some awful pieces of code, running now in production that you’ll have to take a look and fix (or, more probably, rewrite and hope it doesn’t break the system).

The Good Parts

The learning curve

Due to it’s simplicity, PHP can be taught and learned really fast. If you write an html form that posts in a script that saves in a database, well… you code in PHP! Yey! Go tell your mama!

The Comunity

With so many people programming in PHP, it’s community is vast and generous. You can find answer to all your questions on foruns, or on stackoverflow.

And since it’s so damn simple, you can make the most stupid question of the world, and no one will call you “dumb ass”

The Borg Factor!

( If you aren’t a fan of Star Trek, Borg is a race that grows and expands indefinitely, absorbing anything in the way… yes, I’m a big nerd )

Php is always growing, both language and comunity. The secret to this is simple: almost every server, anywhere in the world can run PHP. I think every hosting company put php as default. Damn! Php can run as an Apache Server module. All this with minimal configuration… run apache with the php module, copy a php file into the server root and it works. Seriously, you can’t get better than that.

And the language, as is…

As said before, Php is pretty simple and straightforward. A variable starts with “$” and can be of any type (and can change type in runtime), even a function. You can run without defining classes and objects (non-OO aproach), or (since php 5) build your entire system over the OO perspective. Therefore, a VERY flexible language.

PHP has a lot of frameworks to do the job, cake, zend, magento, etc (just to name some of fullstack). Either you want some “already made” tools, or some magical code that makes the whole system work, php got it. You can spend a good time, just to find the best framework for your need.

And the last comment: Composer. Now Php got a (real) dependency manager for your project. For a old school php dev as myself, that’s something really cool \o/


Well, in the end it’s just the “right tool for the right job”, and php is not a double-clawed hammer. 

Given the scenario PHP is the best tool to be used. Give it a try and let’s talk about it later

Here we go!

Hello there!

As aways, this first post is to introduce myself and what this blog is all about.

My name is Allan and I’m a software developer since 2008. I graduated in Computer Science and, since then, been learning my own way in the computer industry.

When I stop to think about it, I realize it’s not even a decade since I began. There are LOTS of things that I don’t know yet, but I’ll try to show some cool things and thoughts here.

What you’ll see here

I’ll keep my posts about technology, project management and the human side of computer science.

Since english is not my mother language, I hope you forgive me for my mistakes (and correct me when you see something odd). I guess we’re all learning 🙂 .

Languages

I don’t like to be held to just one language. For the moment, the tools that I use to solve my problems are basically Java, Python and PHP. But I’m always searching for some new code adventure.

Contact

If you have any problem with a post, you can comment or get in touch by these:

Twitter: @allanvital
Email: allan.vital@gmail.com