Posts | BackEndTea

Immutable by default: How to avoid hidden state bugs in OOP

Mon, 18 Aug 2025 00:00:00 +0000

One of the harder bugs to debug are bugs dealing with mutable data. A while ago we ran into a problem where the last_modified date of entities was always 1 day off. See if you can spot the issue in the code below:

class MyEntity {
 public function __construct(public DateTime $lastModified) {}
}

function saveEntity(Notifier $notifier) {
 $now = new DateTime();
 $entity = new MyEntity($now);
 $notifier->notifyAt($now->modify('+1 day'));
}

The problem here is that the $now->modify() call mutates (changes) the data of the original object. Since both $now and $entity->lastModified point to the same DateTime object in memory, the $entity->lastModified is also updated.

If we had used DateTimeImmutable, the modify() method would have returned a new object instead of changing the original. The original DateTimeImmutable would stay untouched, meaning $lastModified wouldn’t be updated.

In this example these lines are close to each other, but in reality the entity, and the notifier code were in entirely different places, yet they still used the same DateTime object, thus causing the bug. This cost us multiple hours of debugging, and is why I default to immutable objects.

What does (im)mutable mean?

Immutable means your data cannot be changed, and mutable means it can be changed. When talking about immutability there is a difference between deep and shallow immutability. Shallow means the reference itself is immutable, deep means the reference and everything inside is immutable.

Let’s take the JavaScript const and let as an example. const variables are immutable, and let variables are mutable, meaning we can override a let, but not a const

const immutable = 1;
immutable = 2; // TypeError: invalid assignment to const 'immutable'
let mutable = 1;
mutable = 2; // Allowed.

However, const in JavaScript only gives shallow immutability, meaning that we can change the data inside of it.

const immutable = {
 prop: 1,
};
immutable.prop = 2; // Allowed, since we don't change `immutable`, but its contents
immutable = {
 prop: 2,
}; // TypeError: invalid assignment to const 'immutable'

We can (almost) achieve deep immutability with Object.freeze. This marks each property as immutable. However, nested objects can still be modified if they are not frozen as well.

const immutable = Object.freeze({prop: 1});
immutable.prop = 2; // TypeError: Cannot assign to read only property 'prop' of object

const nested = Object.freeze({outer: { inner: 1}});
nested.outer.inner = 2; // Allowed, since the `inner` object is not frozen.

const nested = Object.freeze({outer: Object.freeze({ inner: 1})});
nested.outer.inner = 2; // TypeError: Cannot assign to read only property 'inner' of object

Most languages have ways to achieve either shallow or deep immutability. It’s good to know which one you are dealing with so you don’t make any wrong assumptions.

Advantages of immutable data

Easier to reason about bugs

Mutable nested objects can be silently changed from anywhere in your code, making it harder to find the root cause of a bug.

To illustrate this, let’s look at a more complete version of the example at the beginning of this post. When reading the Persister code, nothing seems out of the ordinary, and when looking at usages of $entity->lastModified, nothing comes up either. You’d have to check inside the Notifier class to see the modification happening, and know its the same instance as the $entity->lastModified.

class Notifier {
 public function notifyAfter(DateTime $now, int $seconds = 24*60*60) {
 $dateTime = $now->modify("+{$seconds} seconds");
 // Schedule the notification
 }
}

class MyEntity {
 public function __construct(public DateTime $lastModified) {}
}

class Persister {
 public function __construct(private Notifier $notifier) {}

 public function persist(DateTime $currentTime, MyEntity $entity) {
 $this->notifier->notifyAfter($currentTime);
 // Persistence logic
 }
}

$persister = new Persister(new Notifier());

// The entity is created with the current time
$now = new DateTime();
$persister->persist($now, new MyEntity($now));

If we had used a DateTimeImmutable instead, then it would have been impossible to change the inner data of the $entity->lastModified from anywhere else. Since the modify method would just return a new instance, rather than change the original data.

Concurrency

When you are using concurrency (for example, with async await in JavaScript), and using mutable data, then it’s easy to create race conditions, where it becomes impossible to reason about the end result of the code. Take the code below, where all promises are changing the data.count value. Since one may take longer than the other, it’s impossible to determine what the end result of data.count is going to be.

const getRandomInt = (max: number) => Math.floor(Math.random() * max);

const executeAfter = (callback: () => void, milliSeconds: number) => {
 return new Promise(resolve => setTimeout(() => {
 resolve(callback());
 }, milliSeconds))
}

const data = {
 count: 0,
}

await Promise.all([
 executeAfter(() => {data.count = 1}, getRandomInt(10)),
 executeAfter(() => {data.count = 2}, getRandomInt(10)),
 executeAfter(() => {data.count = 3}, getRandomInt(10)),
 executeAfter(() => {data.count = 4}, getRandomInt(10)),
 executeAfter(() => {data.count = 5}, getRandomInt(10)),
]);

console.log(data.count); // Impossible to guess what this value is going to be.

Instead we could have each promise return the new count, and use the last value.

const results = await Promise.all([
 executeAfter(() => 1, getRandomInt(10)),
 executeAfter(() => 2, getRandomInt(10)),
 executeAfter(() => 3, getRandomInt(10)),
 executeAfter(() => 4, getRandomInt(10)),
 executeAfter(() => 5, getRandomInt(10)),
]);

console.log(results[results.length - 1]); // Always 5

Moving from mutable to immutable objects may require some refactoring. The changes required will depend on your use case. But usually returning a value instead of mutating a property is a good starting point.

When not to use immutability

While immutability has clear benefits, there are situations where mutability is the better choice.

Performance overhead

Creating new objects instead of modifying existing ones has a cost. Take the 2 classes below for example. Calling $mutable->add() 1 million times takes about 150-170ms on my machine. Calling $immutable->add() 1 million times, takes about 300ms. If every microsecond counts, then the overhead caused by a pattern where you return a new object every time, may not be worth it.

class Mutable {
 public function __construct(
 public int $counter = 0,
 ) {}

 public function add(): void{
 $this->counter++;
 }
}

class Immutable {
 public function __construct(
 public int $counter = 0,
 ) {}

 public function add(): self {
 return new self($this->counter + 1);
 }
}

Short-lived builder objects

If you are using a builder design pattern, then it is generally understood that you are making changes to this object directly, and that you are mutating state. And while you could use an immutable builder, it’s generally okay to go with mutable data, as the builder object is controller and short-lived. It even helps in keeping the created object immutable, as you have done all modifications needed in the builder phase.

Writing immutable objects

Using keywords in your language of choice like const, readonly, or final isn’t the only thing you need to change when writing immutable objects. There are a few patterns that you’ll likely see when writing or interacting with immutable objects.

With instead of set

Mutable objects usually have setters, which directly change the object. When you want to get rid of those, but still need a way to change data, you can use the “with” pattern. A setX method will change x on the object. A withX method will return a new object where X is changed to the new value.

class MyMutableClass
{
 public function __construct(
 private int $count,
 private string $name,
 ) {}

 public function setCount(int $newCount): void {
 $this->count = $newCount;
 }

 public function setName(string $newName): void {
 $this->name = $newName;
 }
}

readonly class MyImmutableClass {
 public function __construct(
 private int $count,
 private string $name,
 ) {}

 public function withCount(int $newCount): self {
 return new self($newCount, $this->name);
 }

 public function withName(string $newName): self {
 return new self($this->count, $newName);
 }
}

Conclusion

The DateTime bug I opened this post with isn’t a rare occurrence, it can happen any time you deal with mutable state. If we had used immutability, the bug couldn’t have existed. That’s why I default to immutable objects, and only use mutability when I truly have to.

Immutability isn’t always the easiest path, but it’s almost always the safer one. So go and make the next class you write immutable. You might never go back.

Do you agree that immutable should be the default? Or do you prefer the flexibility of mutable objects? Let me know in the comments.

If you want to get notified of the next blog post, join the newsletter.

Types are documentation

Fri, 10 Jan 2025 00:00:00 +0000

Adding types to your PHP code base is more than just runtime validation. It helps both developers and static analysis tools, by making the code clearer and easier to understand. Adding types makes code more self-explanatory and gives you a form of documentation.

A function with proper types will help clarify what it is supposed to do. It becomes easier to determine what a function does from reading its signature. Take the following function as an example. Can you figure out what this method does?

function validateUserIsLoggedIn($user);

There are a few ways this function could work. It can return true or false, telling you whether the user is logged in. It could throw an exception if the user is not logged in. It could even return a result object with information. By adding types we know which case we are dealing with, without having to read the function code:

// Returns whether the user is logged in
function validateUserIsLoggedIn(User $user): bool;
// Will throw an exception if the user is not logged in.
/**
 * @throws UserNotLoggedIn
 */
function validateUserIsLoggedIn(User $user): void;
// Will return a result object (or enum)
function validateUserIsLoggedIn(User $user): UserLoginStatus;

By having these types, we know what the method does. If there are multiple ‘versions’ of this method, we can figure out which one is the best for our use case.

But we can do better than just return types. We can specify whether a method expects something has been checked, or if it assumes that already has been done. For example, do we need to check that name passed to the setUserName is not empty, or does the method check that itself. And of course, we get the added benefit, that if we do pass a possibly empty string to setUserName, then tooling like PHPStan will tell us that we made a mistake.

class User {
 // Do we need to validate before passing, or does the method do that?
 public function setUserName(string $name): void;

 /**
 * @param non-empty-string $name
 */
 public function setUserName(string $name): void;
}

By adding a non-empty-string type to the name, we signal that we are not going to check inside of the method that this parameter is empty. It also helps us inside of the function, as we now know that it can’t be empty, so we don’t need to ‘deal’ with that edge case.

Ready to take your PHP code to the next level? Start adding types today and see the difference!

If you want to get notified of my next blog post, join the newsletter

Why I prefer array functions over loops

Mon, 12 Aug 2024 00:00:00 +0000

I’ve come to prefer using array functions over foreach loops in PHP. And while the syntax isn’t as nice as in JavaScript, I still think they work better than a ’normal’ loop.

What are array functions

In PHP you have quite some array functions, each with their own use. For this post I’m talking about the functions that loop over an array, and execute a function for each item in that array.

You can think of functions like array_reduce, array_map, array_filter etc. Each of these functions does a specific thing for each element of that array, and returns a new value.

Why use an array function

Communicate intent

I think the most important factor for me is the fact that with an array function you are communicating your intent. The function call makes it clear what you are trying to achieve. array_filter? You’re filtering your array. array_map? You are changing each element of the array, but keeping its structure. array_reduce? You are changing your array to a new structure.

This means I can see what someone is trying to do by just reading the function call, so that reduces the mental space I need in order to understand the function

Everything happens inside the function

Everything that the array function does happens inside the function. Take a look at the next example. In the loop part, the $newNumbers is not part of the loop, and lives above it. While in this example its not that big of a deal, I’ve seen the declaration of the new variable being miles away form the loop, due to refactors, and other things happening.


$newNumbers = [];

foreach ($array as $key => $number) {
 $newNumbers[$key] = $number + 2;
}

$newNumbers = array_map(
 fn (int $number) => $number + 2,
 $array
);

Less chance for side effects

In PHP we can write 2 type of closures. with the function($parameter) use($uses) and the fn($parameters) syntax. Since the second one only allows 1 statement, the chance of having side effects is much lower already. And in the first example, anything that isn’t passed in as a function needs to be added through the use(). This means we can clearly see what this loop is using. And the use doesn’t pass values by reference (by default). So if you change a variable in the loop, it resets again for the next iteration. And there is no chance of accidentally overriding a variable that lives outside the loop.

$value = 3;
$new = [];

foreach ($array as $value) {
 $new[] = $value +1;
}
// $value is now the last item of the array;

$value = 3;
array_map(function (int $value) {
 return $value + 1;
}, $array);
// $value is still 3;

When not to use an array function

These functions aren’t always the best solution. For example, if you need to do both a filter and a map, then you can do this in a single foreach loop. But if you use array_filter and array_map, this will actually loop over the array twice. While this probably isn’t a huge bottleneck for most applications, if you have millions of elements this can make a difference. For example, looping over 100k elements, where the filter call filtered out 10%, I got a difference of 0.002 to 0.003 seconds in runtime on my machine.

And sometimes you just need the function to manipulate multiple variables, and have the side effects, in that case, using a loop is probably better.

But, if you have a clear case of having to filter an array, change the elements to something else, or having to ‘reduce’ it to another structure, using an array function wil keep it more readable, maintainable, and less prone to bugs that a loop.

If you want to get notified of my next blog post, join the newsletter

Being a software engineer is a lot like being a butcher

Wed, 07 Aug 2024 00:00:00 +0000

I was a butcher for roughly four years before I became a software engineer. (And nowadays, I’m a vegetarian.) And while one of the jobs doesn’t require you to work inside a refrigerator half the time, there are actually quite some similarities in how we keep our work environment clean.

Keeping your work environment clean

Both as a butcher and as a software engineer I see four different cleaning times/types.

The first type is the cleaning as you’re working. As a butcher things will get dirty when you’re working on your workbench, so you keep it clean by throwing out some trash, removing marinade, putting things back where they belong etc. As a software engineer this is kind of the same as the boyscout rule. Where you work on a piece of code, and you add some types, document a function, maybe refactor something to make it more readable etc.

The second one is unplanned cleaning when there is some time to spare during the day. In the butchery I worked they called this a “tussendtijdse opruming or T.T.O.” which is Dutch for an intermittent cleaning. If we had some extra time to spare we would do a bit more thorough cleanup of either a specific workbench, or of the entire place. Where we clean it up a bit better, to make sure we stay in line with health and safety regulation. This would also allow us to work faster again after the cleanup, as we had more space to work, everything was where it was supposed to be etc.

As a software engineer we have this as well. Sometimes you have a litte bit of time to spare during a sprint, where you can do a bit of a larger refactor, maybe replace a deprecated library with a new one etc. While not necessarily planned, these clean ups are a bit larger than just touching a few things while you’re working on it.

The third type of clean up as a butcher was the end of day cleaning. Here we would do a thorough clean of the place, using water and soap to clean up all the workbenches. We would hose down everything (twice), and get everything set up again for the next day.

As a software engineer we don’t have a “closing” hour, but we do have the end of a sprint or a period. You should always reserve some time at the end (or start) of a sprint, to do some cleaning. You can create tickets as you’re working on something for potential clean ups. Maybe a business critical piece of software is extremly confusing to read. So if you have some time you can create good tests for it and rewrite it to be more readable. This is the type of work that is planned, but shouldn’t take more than a few days to complete.

The last type of cleanup as a butcher was the type of cleaning we would do once or twice a year. During the time of year when it was less busy, we’d take time to maybe clean the freezer. We could scrape off all the ice, do a full clean, and throw out anything that looked suspicious. We’d also clean parts you normally couldn’t reach, like the wall behind a storage rack.

In software engineering, this is the type of cleanup that takes a longer time, and is planned in advance. This can be things like replacing a library that is at the core of your business, which has a large impact to replace. Things like these can take an entire sprint, but are needed to keep your codebase up to date, and keep it functioning for years to come.

If you want to get notified of my next blog post, join the newsletter

Break the rules! (but only if you understand them)

Wed, 31 Jul 2024 00:00:00 +0000

There are a lot of ‘rules’ in software engineering. Don’t copy-paste your code, don’t use goto, your code should be easily understandable, and many more. Generally these rules make sense, but every once in a while someone will have a hot take, and post about how you should copy-paste your code everywhere, how goto is the greatest invetion ever, or how variable names should be shorter.

I’d like to advocate for a middle ground, and say that ‘it depends’ (which is true for almost anything in software). While most of the time you shouldn’t copy-paste your code, there are cases where it makes sense. And while goto is generally a bad idea, the symfony code bases uses it.

When can you break the rules

Personally I try to only break a rule if the following are all true:

I understand why this rule is in place

With the example of copy-pasting code, I’d like to think at this point I understand why this rule is in place. If you copy-paste code all over your code base, you reduce maintainability. You make it harder to make a change, instead of having to update it one place, you have to update your code at every place you copy-pasted it to.

There is an actual benefit to breaking this rule

Often people will break a rule, just because it easy at that moment. Copy-pasting your code is easier than making a good abstraction to re-use. But sometimes there are benefits. For example, sometimes you want to copy-paste your code if there is not ‘right’ abstraction. If 2 pieces of code to the same thing, but that by coincidence, then an abstraction doesn’t make sense.

The benefits outweigh the downsides

Even if there is a benefit to breaking the rule, it has to outweigh the negatives. And this really depends on the goals for your code base. For example if you’re writing a piece if code that gets executed a million times per request, then, the performance of that piece of code may be more important than the readability. But it comes down to what is more important for that piece of code.

Closing thoughts

A lof of the times I’ll decide not to break a rule at because there isn’t an actual benefit (besides it just being easy), or because while there is a benefit, it doesn’t outweigh the downsides. The first check is mostly there to check if I actually know what I’m doing. Especially if I’m doing something new, like trying another language, or working in a new paradigm, then understanding the rule comes into play. If you’re just starting out with something, try to stick to the rules, until you understand the up and downsides of that rule.

What rules do you break sometimes, and how do you determine which rules to break? Let me know in the comments!

If you want to get notified of my next blog post, join the newsletter

PHP 8.2 changes to iterator functions I missed

Tue, 30 Jul 2024 00:00:00 +0000

This week I learned of a small change that got introduced in PHP 8.2 that I completely missed. The iterator_*() functions got an update, which make them actually usable when dealing with the iterable type. Now this is only really relevant if you are dealing with iterables that might not be arrays, but you need to use them as an array.

The big change is that a function like iterator_to_array, it now accepts the iterable type, rather than just \Traversable. For some code I’m working on it means I can refactor a lot of code to remove is_array checks all over the place.

// Pre PHP 8.2:
function getAllStringLengths(iterable $input): array {
 if(!is_array($input)) {
 $input = iterator_to_array($input);
 }

 return array_map(strlen(...), $input);
}

// Post PHP 8.2
function getAllStringLengths(iterable $input): array {
 return array_map(strlen(...), iterator_to_array($input));
}

If you want to get notified of the next blog post, join the newsletter.

"Just use Psl for that"

Tue, 23 Jul 2024 00:00:00 +0000

Something I’ve been hearing myself says in every project that I have installed Psl in, is “Just use Psl for that”. And while some bits and bobs like array_find are getting into PHP 8.4, the library does so much more. In this post I’ll go over the bits that I use often, and maybe convince you to give it a try as well.

What is Psl

Psl is a php composer package that can be found right here.

To quote the Psl readme:

Psl is a standard library for PHP, inspired by hhvm/hsl. The goal of Psl is to provide a consistent, centralized, well-typed set of APIs for PHP programmers.

And that is what it does, from async operations, to working with the filesystem, to hashing, it provides a clean api, that is consistent and well typed, and integrates well with static analysis tools like PHPStan and Psalm.

Useful Psl parts

Type

In my opinion the Type namespace is probably the most useful part of Psl. Very often we have to validate some data that comes from some input where we are (almost) guaranteed what type it is, but we can’t be 100% certain. Here the Type namespace of Psl comes into play. Let’s say you have a specific structure you expect to receive from and API, and you want to validate that it is in fact correct. With Psl that would look something like this:

use Psl\Type

$userResponse = fetchUsers();

$userResponse = Type\shape([
 'users' => Type\vec(Type\shape([
 'id' => Type\int(),
 'name' => Type\nullable(Type\string()),
 'email' => Type\string(),
 'roles' => Type\vec(Type\string())
 ])),
])->coerce($userResponse);

Now after this line, you are guaranteed that $userResponse is an array, with the key users, which is a list of arrays, containing the keys specified, with their respective types. This is a runtime check that will error out if the type does not match. It also helps PHPStan and Pslam understand the structure of the data.

Each function in the Type namespace returns a Type object that you can use to either coerce or assert your data. assert checks that your data 100% matches, and then return that. Meaning that Type\int()->assert('1') wil fail. coerce will try to coerce your data, meaning that Type\int()->coerce('1'), will return the integer 1. There is also the matches function, which returns a bool of whether your type matches.

Dict, Vec & Iter

These three namespaces have a lot of overlap. Dict is for Dictionaries, Vec for lists, and Iter for general iterables. They contain all kinds of usefull array/iterable functions that you need, that aren’t present in PHP natively, or require multiple function calls.

For example if I want to filter out all elements that are 3 or smaller, but keep my list as a list, I need to do an array_filter, and then don’t forget to wrap it in an array_values. With Psl, since the Vec namespace is about lists, it makes it a list, regardless of whether it was a list before, or even if it was a Generator before, without any extra steps required.

use Psl\Vec;
// Native
array_values(array_filter($myArray, function (int $input) => $input > 3));

// Psl
Vec\filter($myArray, function (int $input) => $input > 3))

One function I tend to use a lot from the Dict namespace is Dict\reindex. What this does is reindex an array given a callback. Which you can do with something like array_reduce, or in a foreach loop, but having a dedicated function to do some makes the code a lot more readable in my opinion.

$users = [
 ['id' => 42, 'name' => 'foo'],
 ['id' => 24, 'name' => 'bar']
 ];

 $indexedUsers = Dict\reindex($users, fn($user) => $user['id'])
 // $indexedUsers is now:
 [
 42 => ['id' => 42, 'name' => 'foo'],
 24 => ['id' => 24, 'name' => 'bar']
 ]

And these few namespaces aren’t even scratching the surface of the functions this library provides. Although I currently don’t have to deal with Async operations in an existing codebase, if I needed to, I would probably check out Psl first.

If you want to get notified of the next blog post, join the newsletter.

Why you should be typing your arrays in PHP

Wed, 17 Jul 2024 00:00:00 +0000

We have been able to natively type parameters of methods and functions in PHP for quite some time. In basically any version of PHP you should be running we can do something like the following:

function getNames(array $input): array {
 return array_map(function($item) {
 return $item->getName();
 }, $input)
}

Basic array types

This however tells us very little about the types that we are dealing with. When specifying a type as just array or iterable, or something alike, all you are saying is that it is an array, but you really want to specify that it is an array containing a specific type. If we rewrite the previous example to be more specific, we can do the following:

/**
 * @param array<User> $input
 *
 * @return array<string>
 */
function getNames(array $input): array {
 return array_map(function($item) {
 return $item->getName();
 }, $input)
}

If we add types like this we now understand we are getting an array which contains User objects, and we are returning an array of string. There are multiple benefits to doing so.

Benefits of typed arrays

IDE hints

The first benefit is the fact that your IDE will now understand what kind of array this is, and thus be able to give you autocompletion inside the function. It will also help you when trying to find the usages of getName() on the User class, as now your IDE, knows it is that method that is being called.

This will also help you with refactor tools, for example if you are renaming the getName method, an IDE like PHPStorm will be able to also update this method call. You’ll also get errors in your IDE if you pass in the wrong type of array, or if you try to do something that’s not possible with the returned array of strings.

Static analysis tooling

The second benefit comes from the usage of static analysis (SA) tooling (like PHPStan). SA tools will understand what you are doing, and give you error in your pipelines if you are doing something that shouldn’t be possible. The more types you use in your codebase, the better understanding it has of your code, the better it can help you detect issues before they hit production.

In PHPStan adding type declarations becomes a requirement on level 6. This may be hard to add everywhere in legacy code bases, but I would highly recommend upgrading to at least this level and generating a baseline. This will require you to at least add these types to any new code you are writing

Developer understanding

Most likely you aren’t the only one working on your code base. If the next developer comes along and sees the getNames method, they first have to figure out what the input is, before they can figure out what they are supposed to do with the method.

I can’t count the amount of times I looked at a method and thought “I have no idea what’s going on here”, because there were either no types, or just array as a type. I would first have to search through the code base to see where the method is called, before being able to understand what the parameters where and what the return values where.

Advanced array types

With modern PHP tooling, we can however, go much further than just doing @param array<Type> $value. We can do lists, specify array keys, or make sure it is not empty

The list type

We can for example specify something is a list.

A list is an array, where the keys start at 0, and each time they increment by one. The list type can be especially useful when converting to JSON for example, as that will make it an array, instead of an object. If you create an array without keys, it is always a list, but you can also specify the keys if needed

$list = [1, 2, 3];
$alsoList = [
 0 => 1,
 1 => 2,
 2 => 3,
];
// Keys start at 1, not at 0, so not a list.
$notList = [
 1 => 1,
 2 => 2,
 3 => 3,
];
// Has a string key, so not a list.
$alsoNotList = [
 'foo' => 'bar',
 1,
 2
];

The list type doesn’t exist natively, but you can use it like so, and a tool like PHPStan will help tell you if you aren’t passing a list

/**
 * @param list<User> $input
 *
 * @return list<string>
 */
function getNames(array $input): array {
 return array_map(function($item) {
 return $item->getName();
 }, $input)
}

Specifying keys

Nowadays, we can even specify that we expect a specific set of keys, or just what type the keys should be. With the following notation we specify literal keys: array{key: Type, key2: Type}. This also comes with IDE autocompletion for those keys, and all other previously mentioned advantages. If we want to specify just the general types, for example if the keys are strings, we can do that as follows: array<string, Type>. Generally the notation for arrays is array<KeyType, ValueType>. If we only pass one type, it is the type of the value.

/**
 * @param array{begin: int, interval: int} $data
 *
 * @return int
 */
function transform(array $data): int {
 return $data['begin'] + $data['interval'];
}

/**
 * @param array<string, mixed> $data
 *
 * @return string
 */
function concatKeys(array $data): string {
 $output = '';
 foreach($data as $key => $v) {
 $output .= $key;
 }
 return $output;
}

Non-empty types

We can even go as far as to specify the array should not be empty. you can do that by using non-empty-array (or non-empty-list), instead of just array (or list). If we take the concatKeys function as an example, we could define it like so, to make sure it is never empty. And we also specify the returned string will never be empty.

/**
 * @param non-emptyarray<string, mixed> $data
 *
 * @return non-empty-string
 */
function concatKeys(array $data): string {
 $output = '';
 foreach($data as $key => $v) {
 $output .= $key;
 }
 return $output;
}

And once again, this provides better understanding for SA tooling, your IDE, and other developers.

Conclusion

Adding types to your arrays (and just adding types in general) helps you when using an IDE, or static analysis, and helps other developers in your team to understand your code. It should help you produce less bugs, and make your code cleaner and more readable.

These types help you document the intent of a function and how it should be used. And even better, with modern day tooling, it even shows you if you are doing something wrong.

Are you using (array) types in your PHP codebase, and do you think they are usefull? Are you already using more advanced types like the list and non-empty-array. Let me know in the comments!

If you want to get notified of the next blog post, join the newsletter.

How to fix "connect to host github.com port 22: Operation timed out"

Tue, 09 Jul 2024 00:00:00 +0000

The problem

This weekend I was at a holiday resort. In the morning I wanted to do a little bit of work, and encountered the following error when I tried to pull a repo.

$ git pull
ssh: connect to host github.com port 22: Operation timed out
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

At first, I assumed my internet had a hick-up, so I just ran the command again, only to be greeted by the same error message as before.

As it turns out, the holiday resort I was in was blocking basically anything that wasn’t normal http requests, which use port 80 and 443 for http and https.

Connecting ssh over another port

Thankfully both GitHub and GitLab offer a solution for this problem. Both have an alternative ssh server, that you can ssh to on port 443. To use it, you need to change your ssh config, which you can find in ~/.ssh/config.

Add the following bits in your ssh config for either GitLab or GitHub. The Host <url> part tells us the next bit of config is for that host. The Hostname part makes your ssh use this for host instead of the original one. The User key give it the user (which is git by default, but we specify it to be sure. And finally the Port is the bit that helps us out, as we will now ssh over port 443. If you attempt to just change the SSH port to 443, but keep the normal host, that unfortunately won’t work, as the normal servers only accept ssh over port 22.

# GitLab config
Host gitlab.com
 Hostname altssh.gitlab.com
 User git
 Port 443

# GitHub config
Host github.com
 Hostname ssh.github.com
 Port 443
 User git

Once your holiday is over, and your back home, you can just remove the config you added, and you’re back on the original GitHub or GitLab servers.

If you want to get notified of the next blog post, join the newsletter.

What is PHP's declare(strict_types=1); and why you should use it

Thu, 04 Jul 2024 00:00:00 +0000

You may have had people tell you that you should use declare(strict_types=1); in your PHP files, but what exactly does it do, and why should (or shouldn’t) you use it?

History

A long, long time ago PHP didn’t have scalar types, or even return types. In PHP 7.0 both of these got introduced. In 5.6 and before we had to use annotations, and couldn’t add it in the language. (If you still see code like this it’s probably from that era.)

/**
 * @param string $input
 * @return string
 **/
function foo($input) {
 var_dump($input);
 return $input;
}

Nowadays we can use scalar types, return types, property types. We even have unions and intersection types now. Which means we can write the same code from before like so.

function foo(string $input): string {
 var_dump($input);
 return $input;
}

Using declare(strict_types=1)

But what do you think happens if we then call this method with an integer, or a boolean like so? Will this give us a TypeError, will it cast the type or will it stay the original type?

foo(3);
foo(2.5);
foo(true);

To determine what happens we need to look at the top of the file that is doing the calls. If we see declare(strict_types=1), then a TypeError will be thrown when we run this code. If you see nothing related to strict_types or declare(strict_types=0); Then the type will be cast, and we would get the following output:

string(1) "3"
string(3) "2.5"
string(1) "1"

Why you should use declare strict types

Unintended type casts can result in unexpected behavior. If we have a function that takes a boolean, but we pass it a string, any truthy string will become true. So if you get user input from a JSON POST for example, and you pass the incorrectly typed data, it will always be interpreted as true, and may cause hard to uncover bugs.

However, if you use declare(strict_types=1); everywhere in your code base, you instead have a TypeError being thrown, causing execution of your code to stop, and (hopefully) your monitoring going off. So while it may break your website for a bit, it will help you figure out the issue much faster and reduce weird behaviour.

When something breaks you want it to break loudly. It’s generally better to have an exception or error being thrown when unexpected issues arise, rather than ignore it and continue your program, even if everything is going wrong. Because before you know it, you have wrong data in the database, or even emails being sent to customers when it wasn’t intended.

For example, we had an older part of the code base that handled the login flow. The login method looked something like this, and had no types as it was originally written for PHP 5.X. Now the types here are actually wrong, for password a string (namely the password itself), was provided if there was one, and false was passed if it was a login without a password (a log in with google button for example).

/**
 *
 * @param string $method
 * @param string $userName
 * @param bool $password
 */
function login($method, $userName, $password = false)

At some point we had to do some refactors in this part of the codebase, and decided to add types. And suddenly, no one could log in anymore with a password. Any password they passed was just cast to true, and then later when that was checked with password_verify, the 2 passwords didn’t match.

function login(string $method, string $userName, bool $password = false)

If we had declare(strict_types=1); at the top of this file, we would’ve had a TypeError. This does mean customers would’ve seen an Oops something went wrong! type of screen. It also would’ve meant our alerting would’ve gone off, and the change would’ve been rolled back, while we looked into the issue. With a stack trace it would’ve been easy to spot where this went wrong, and we could have updated the type like so. But because we didn’t have that, we had to spend hours debugging why customers reported not being able to log in.

function login(string $method, string $userName, bool|string $password = false)

Caveats of strict_types in PHP

As I mentioned earlier, you have to look at where the code is calling from.

Let’s say we have 2 files, functions.php and index.php, where one has declare(strict_types=1), and the other doesn’t. In this case, no TypeError is thrown, as the code that is executing the takesString, isn’t using strict_types.

<?php
//functions.php
declare(strict_types=1);

function takesString(string $input) {}

<?php
//index.php

require_once __DIR__ . '/functions.php'
takesString(3); // no error

But if we now turn this around and make index.php strictly typed, then we get a TypeError, this is because the code that is executing takesString is now in strict_type mode.

<?php
//functions.php

function takesString(string $input) {}

<?php
//index.php
declare(strict_types=1);

require_once __DIR__ . '/functions.php'
takesString(3); // throws a TypeError

How to use declare(strict_types=1) in PHP

If you decide to use strict_types in PHP, then I would advise to use it everywhere in your code base. When you are using tools like php-cs-fixer or php code sniffer, then you can add checks to make it required and add it to the whole code base at once. That may not always be possible for a legacy project. In that case you could use a tool like PHPStan to make it required, and add all files where you currently can’t add strict types to the baseline. See this rule on how to do that.

If you want to get notified of the next blog post, join the newsletter.

Adding PHPStan to a legacy project

Thu, 27 Jun 2024 00:00:00 +0000

PHPStan

PHPStan is a static analysis tool for PHP. It helps you find problems in your code base, like passing wrong types, calling methods that don’t exist, detect wrong annotations, and so much more.

I would argue you shouldn’t be writing PHP code these days without having a static analyzer to help you. It reduces the errors you create, and speeds up your development significantly.

How to add PHPStan

1: Composer install

The first thing you’re gonna have to do with PHPStan is to install it. You can run the following command to install it. You will need PHP 7.2 to run and install it, but it can analyze older code if needed.

$ composer require --dev phpstan/phpstan

2: Configuration

You can get started with a really basic configuration file, which is called phpstan.neon. The relevant config goes below parameters. We’re starting with level 0, which is the lowest level of PHPStan.

At level 0, we are doing very basic checks, like checking for unknown classes/methods, the wrong amount of arguments passed to a method.

# phpstan.neon
parameters:
 level: 0
 paths:
 - src/
 - tests/

3: Your first PHPStan run

Once you have PHPStan installed, and a config file set up, you can run PHPStan like this

$ vendor/bin/phpstan a

Now if your project is set up correctly, and you have no problems at level 0, you should see an error like this:

 [OK] No errors

But, chances are that your code might not be understandable by PHPStan just yet, and you get errors.

Unknown classes due to custom autoloader

If you have a custom autoloader set up, you can tell PHPStan to use that like follows. This will help PHPStan find the relevant classes. (This file will be executed by PHPStan, so make sure it has no side effects).

parameters:
 bootstrapFiles:
 - custom_autoloader.php

If you want PHPStan to know about a certain file or directory, but not analyze it, you can add them as follows. This will let PHPStan find all the classes, functions etc from these directories, but not report errors in them.

parameters:
 scanFiles:
 - ThirdParty.class.php
 scanDirectories:
 - ./my_classes/custom

And finally, if you have a phar that gets included, that has classes, you can make that phar a bootstrap file as well. This lets PHPStan find its symbols. If you have a PHPUnit phar for example, you can add it like so:

parameters:
 bootstrapFiles:
 - phpunit.phar

4: Adding it to your CI

If you already have a CI set up, adding PHPStan shouldn’t be too difficult. I would make sure you have no errors on level 0 before doing so. If there are too many errors check the part of this post about the baseline. You can run vendor/bin/phpstan a in your CI, like you would a PHPUnit or any other tool.

At the level 0 we just created, PHPStan won’t have much impact, but it’s good to have it running in your CI already. This makes it so that once we start making PHPStan more strict, we make sure that the rest of your team needs to follow the rules from PHPStan.

5: Increasing the PHPStan levels

At this point you have PHPStan running in your CI, which means that the real work begins. Depending on your code base, just level 2 can already be a difficult task. If at all possible, try to fix all the errors of a level and then move onto the next one. Do this until you start running into too many errors to fix at once. Depending on your code base this can be level 2, 5, 6 or even higher.

Bumping the level is as easy as changing the level value in your config file. Or you can pass the --level X flag when you run PHPStan, to use that level for a single call.

6: Adding a baseline

At this point you have PHPStan in your CI, but the amount of errors to fix at once is becoming too much to fix at once. Now you can start introducing a baseline. In PHPStan the baseline is a file containing all your errors, which PHPStan will ignore for you. Now ignoring all the errors may seem counterintuitive at first. You want to fix the problems PHPStan is finding for you, so why ignore it?

The baseline isn’t about fixing all the errors you currently have. Instead, it makes sure that your new code follows the rules you set up. By using a baseline we can increase the checks PHPStan is doing, without having to fix all our code at once. So all the new code you write, and code you change, needs to adhere to the level you have set.

Once you have a baseline set up at your level, you can fix errors in whatever time frame is required. You could choose to fix errors when you touch a file, or maybe take a bit of time every sprint to fix some errors. Depending on how far along you are, you can even bump your level all the way to max level and make a bigger baseline. This will make all your newer code much more type safe, without having to fix all the errors at once.

Getting your team onboard

Usually the biggest problem in adding a tool like PHPStan isn’t adding the tool itself, it’s getting your team onboard with these new demands your tool makes. This is why you can’t just add it at max level with a massive baseline and expect all your colleagues to completely change the way they write code. Instead, you’re going to have to phase it in step by step.

I like to start by adding PHPStan at level 0 or 1 and having it run in CI. This will get your coworkers to get used to the tool occasionally telling them about a big error they made. While this is running you can show them how to run PHPStan for just a specific file, or even for only the changed files in an MR, at a bit of a higher level to catch even more errors. When you feel like they are becoming more familiar with the tool, you can bump it again in CI and fix more issues.

Running PHPStan on only changed files at another level is as easy as running:

vendor/bin/phpstan a $(git diff --name-only) --level <desired level>

Depending on how the adoption in your team goes, you may even have other colleagues asking to increase the PHPStan level, so they can find more errors. And once you’re at a suitably high level you could even consider running PHPStan on bleeding edge. By doing so you are becoming forward compatible with new major versions of PHPStan, and finding even more errors that aren’t the default yet.

Conclusion

If you are introducing PHPStan to your toolkit, be sure to go slow and steady, fix the errors at a lower level, and slowly increase that over time. Do this until your team has adopted it, and knows how to work with it. You can show your colleagues how to properly use it, and show them the advantages of PHPStan, to increase the speed at which the adoption happens.

If you want to get notified of the next blog post, join the newsletter.

What ive been reading: Deep work by Cal Newport

Wed, 26 Jun 2024 00:00:00 +0000

Deep work

I finished reading deep work about a week ago. If you want to become a mythical 10x developer (or 10x anything really), this book provides you with the tool on how to succeed in just that.

A lot of us fall into a trap of doing mostly, what this book calls, shallow work. We have meetings that aren’t really important, we do meaningless tasks, and spend half our day in meeting that could’ve been avoided. The book first focuses on why deep work is important, and why so many of us are doing so little of it. It shows why your organization can even stop you from performing meaningful deep work, and how to start dong more deep work.

What is deep work

Personally I would define deep work as deeply focused work. It is the ability to focus deeply on something for longer periods of time, without interruptions. We’ve all had work days when every time you are getting into the groove, something pops up, and you have lost your focus. Whether it’s a colleague with a question, an email that needs an immediate response, or something else entirely. Every time we lose this focus, we need a lot of time to get it back again. Only to have go into the next meeting.

Deep work is about saying no to certain requests, not going to the meetings that aren’t needed, and focusing on doing the work that you actually want/need to do. The book argues that most emails don’t need a response within a minute and most questions on slack don’t need to be responded to right away.

Deep work as a developer

Deep work as a developer means increasing your productivity by a lot. Your first order of business would be to not always be available for questions. It is okay to set aside chunks of time to focus on your work, without checking slack, email or your phone every minute. Of course, you can’t just start doing this right away, and you’ll need to make sure your team knows you can’t be reached at every minute of the day like before. The book also goes into detail on how to set this up for yourself.

Deep work is now sitting on my bookshelf, but I’m sure it’s a book that I open up again from time to time to remember how to deal with certain types of shallow work, and how to become more efficient with my time. So I would definitely recommend this book to any developer looking to improve their career. You can buy deep work on amazon.

If you want to get notified of the next blog post, join the newsletter.

5 Tips for better code reviews

Fri, 21 Jun 2024 00:00:00 +0000

A big part of software engineering is code review. Good code reviews will help your code base become better and help you and your colleagues learn from each other. Let’s take a look at some code review best practices and tips!

1: Have your tools check the code for you

The best way to have good code review is to reduce what needs to be reviewed as much as possible. When reviewing code, I shouldn’t have to check whether you’re indenting with tabs or spaces, a tool should do that. I shouldn’t have to check if the tests pass, that should be done in CI.

This doesn’t mean we need a passing CI before we can do code review. Sometimes we know there is an error happening, but we are looking for some early input to see if we are heading in the right direction.

2: Ask questions instead of demanding changes

This is especially important when dealing with junior developers, who might not be as confident about their code as more senior developers. For example, if an input field could be of type email you should not just say Hey, this can be type="email". Instead, ask the other developer why they chose for type text. It might be that legacy users don’t have email addresses, but usernames. Or you might be right and it could be of type email, in which case they can change it. But if you just demand a change to be made, the dev might just think you know better and change it without pushing back.

3: All devs should code review

Some folks think that only seniors should do the code review, but I always encourage junior developers to do code review as well. It’s a great way to learn about the codebase, and about the decisions someone made while coding. It does mean that seniors will have to spend a bit of extra time to answer the question that may pop up during the review, but it will increase the skills of the juniors on your team a lot.

4: Review things that matter

As developers, we sometimes get stuck on small unimportant things, and we can spend three days arguing whether a variable should be called $password or $userPassword. Try to avoid these kinds of things while doing code review. Instead, focus on the bigger things in a code review, look at bottlenecks that may cause performance issues, and make sure the code will be readable by the next person to look at it.

5: Use FTI or NIT

If something is more of an off-hand comment, or doesn’t need to be changed, you can use FTI: or NIT: in front of your comment, which stand for Free To Ignore and Nitpick respectively. You can use this for comments that don’t matter that much, and you could see as nice to haves. Maybe someone is refactoring a legacy class, but didn’t add types everywhere. This isn’t a dealbreaker since we are already doing some boy scouting, but more improvements could be made.

In that case you could leave a comment like: “FTI: There are still some types missing on the methods, maybe we can add those as well?” Because of the FTI:, the developer knows they don’t have to do this. Generally speaking no changes need to be made, if you have a reason why the current solution is better than what the reviewer is asking. But with FTI:, you give the developer permission to ignore it, or just not make the change without having to reply.

If you want to get notified of the next blog post, join the newsletter.

Analyze your tests with PHPStan

Mon, 10 Jun 2024 00:00:00 +0000

Hopefully you are already using PHPStan to analyze your source code. But you shouldn’t stop there, in fact, you should also make sure you are analyzing your test code.

Why analyze your tests

If you are already using PHPStan to analyze your source code, then you probably see the value in static analysis. The same reasons for analyzing your source code are true for your tests. You can find bugs without running your code, and it tells you of potential errors.

There is one difference between your test and source code however. Which is that your test code is (probably) always executed in your CI. Meaning that if an error is guaranteed to occur you would notice it while running your tests. But if you have a larger project where the tests may take a long time, the static analysis might be faster.

There are more reasons to analyze your tests though. PHPStan will also help you find assertions to make your tests more clear, and detect assertions that are always true.

Useful extensions

The best extension to use if you are analyzing your tests is the PHPstan PHPUnit extension.

This will detect that the following assertion is useless, as the value is always true. This is of course a simplified example, but I’ve seen conditions like these in my tests before.

$this->assertTrue(true);

Another great feature of the plugin is that it will tell you if you can use more specific assertions. You can read why that is usefull right here.

So for example it will tell you to make the following change:

- $this->assertSame(true, $var);
+ $this->assertTrue($var);

Conclusion

If you aren’t analyzing your tests yet, start doing so today. Simply add your tests folder to your PHPStan config and start today. You can either add all your errors to the baseline, and fix them at a later point, or perhaps fix them today.

If you want to get notified of the next blog post, join the newsletter.

How to create a random string in PHP

Tue, 04 Jun 2024 00:00:00 +0000

There are a lot of ways to generate a random string in PHP. In this post we’ll take a look at two options we have, and how to use them.

The Randomizer

The Randomizer class (introduced in PHP 8.2) is arguably the best way in OOP PHP to use randomness, and should be used whenever possible. It comes with 2 methods for creating random strings. Where getBytes will give you a random set of bytes of the specified length, but characters may not be printable, or valid UTF-8. The second one, is the one you’ll likely use most of the time, which allows you to specify what characters you want in your output string. If you specify a character multiple times in the $string variable, it is more likely to occur.

// Introduced in PHP 8.2
public getBytes(int $length): string
// Introduced in PHP 8.3
public getBytesFromString(string $string, int $length): string

How to use the Randomizer

The randomizer class takes an Engine interface as an optional argument. If not provided, PHP will use the Secure variant. So lets write a class that generates a random subdomain for a website.

use Random\Randomizer;

class SubDomainBuilder
{
 private const KEY_SPACE = 'abcdefghijklmnopqrstuvwxyz0123456789';

 public function __construct(
 private Randomizer $randomizer,
 ) {}

 public function createsubDomain(string $domain): string
 {
 return sprintf(
 '%s.%s',
 $this->randomizer->getBytesFromString(self::KEY_SPACE, 8),
 $domain
 );
 }
}

$builder = new SubDomainBuilder(new Randomizer());

echo $builder->createsubDomain('example.com') . "\n";

In this SubDomainBuilder class, we have the createsubDomain method, where we use the getBytesFromString method of the Ranomizer. We pass it a keyspace of a through z, and the numbers 0 to 9, so we get a sub domain which is only lowercased letters and numbers. Running this script will output something simmilar to x0elme6k.example.com.

One of the great things about the Randomizer class is that you can provide an engine, and can seed that engine. This will make the results deterministic, and allow you to use the randomness in unit tests.

For example, if we change the using of the class to the following, we’ll get the same results every single time. (nleajcf2.example.com). This means that we can use a seeded Engine in our unit tests, to make sure we always get the same results.

$random = new Randomizer(new Mt19937(123));

$builder = new SubDomainBuilder($random);

echo $builder->createsubDomain('example.com') . "\n";

Creating a random string before PHP 8.2

If you are on PHP 8.2 or higher you should use the Randomizer. However, if you’re stuck on an earlier version of PHP, and want to generate a random string, you can create your own Random generateor which uses the random_int function as follows to generate your own random strings. The class would roughly look as follows. You could even define often used key spaces in this class to re-use.

class RandomGenerator
{
 public function getBytesFromString(string $keySpace, int $length): string {
 $keySpaceLength = strlen($keySpace);
 $randomString = '';
 for ($i = 0; $i < $length; $i++) {
 $randomString .= $keySpace[random_int(0, $keySpaceLength - 1)];
 }
 return $randomString;
 }
}

Conclusion

In newer PHP versions is has become surprisingly easy to generate random strings. If youre on version 8.2 or higher, the randomizer is the way to go. If youre a bit earlier than that, you can still use the random_int function to create safe random strings.

If you want to get notified of my next blog post, join the newsletter

Mastering PHPUnit: Using Mocks and Stubs

Thu, 30 May 2024 00:00:00 +0000

This is part 4 of a series on PHPUnit testing, you can find part 3 here. In this post we’ll focus on the use of mocks. We’ll cover why we need them, the different types of mocks, and how to use them.

Why do we need mocks?

A mock is a stand in for a dependency that you dont want to (currently) test. The most commonly used example might be the database. In your unit tests, you don’t want to actually access the database. Instead you want to mock the usage of the database, so that you can test your logic without it. The two reasons for this are speed and isolation. To start with speed, unit tests need to be fast. If you unit test takes more then a few miliseconds it is slow. (Don’t take the startup time of PHPUnit into account for this.) In a bigger project you’ll likely have thousands if not tens of thousands of unit tests. If each of them would have to talk to the database, and then reset it afterwards, your test suite would take hours.

Another reason to have mocks is to test your code in isolation. Generally we want our unit tests to test one thing. So we dont’ really care what all the other classes are doing under the hood. Those have their own unit tests to cover that behaviour. Take the next bit of code for example. We fetch some data, and make sure the status is OK. In a unit test we dont want to use the HttpFetcher class, instead we’ll mock that, and only test the checking of status.

class BuildData
{

 public function __construct(
 private HttpFetcher $fetcher,
 ) {}


 public function build(): array
 {
 $data = $this-fetcher->fetch();

 if($data['status'] !== 'OK') {
 throw new Exception('Status was not OK.);
 }

 return $data;
 }
}

Different types of mocks

I’ve been talking about mocks so far, but in PHP we can differentiate 2 types, which are Mocks and Stubs. The difference lies in what we do with them. On a stub we can configure behaviour. We can tell it what to return on a certain method call for example. On a mock we can also do this, but in addition to that we can also configure expecations. So we can make sure the method gets called twice, or with certain parameters.

If the ‘side effect’ of your method is what you are interested in, then you want to test that using mocks. Otherwise you should use a stub. For example, if you have a method which needs to make an API call with very specific parameters, then you can use a mock to test that that call was made with the right parameters. But if you have a method which needs to get some data from a dependency, then you should just use a stub.

In general i would always use a stub, unless its absolutely necesarry to use a mock. Which would only be if you really need to test that a certain call is made.

How to use mocks

Lets try to write a test for the BuildData class that we wrote a little earlier. Without mocks it would look something like this:

class BuildDataTest extends TestCase
{

 public function testItReturnsData(): void
 {
 $buildData = new BuildData(new HttpFetcher());

 $data = $buildData->build();

 $this->assertSame(['status' => 'OK'], $data);
 }
}

Now this test is gonna fail. The htpt server can’t be reaced, or maybe we need some more set up for that. But we don’t need to talk to the external server, if we use a stub for this. We can create a stub in PHPUnit with $this->createStub(<class_name>). So lets do that:

class BuildDataTest extends TestCase
{

 public function testItReturnsData(): void
 {
 $fetcher = $this->createStub(HttpFetcher::class);
 $fetcher->method('fetch')
 ->willReturn(['status' => 'OK']);

 $buildData = new BuildData($fetcher);

 $data = $buildData->build();

 $this->assertSame(['status' => 'OK'], $data);
 }
}

Now this test will pass. We use the willReturn to configure our stub to return the correct data. (in this case the ['status' => 'OK']) Since here it is not that important how many times something is called, or what it is called with, we dont need to use a mock, and we can use a stub. One example where the amount of times something is called, or with what it is called does become important is with cached calls. For example in the following adapter, we want to test that we only call our api once per id.

class CachedApi implements Api
{
 /** @var array<int, string> */
 private array $cache = [];

 public function __construct(
 private Api $innerApi,
 ) {}


 public function fetchApi(int $id): string
 {
 return $this->cache[$id] ??= $this->innerApi->fetchApi($id);
 }
}

So for this class we could write the following tests, where we use createMock instead of createStub. We now also use the expects to configure how often something should be called. (You can also use $this->never() to configure if something should never be called). And in the first test method we use with, to configure what the method should be called with, since it is relatively important.

class BuildDataTest extends TestCase
{

 public function testItCallsInnerApiOnce(): void
 {
 $innerApi = $this->createMock(Api::class);
 $innerApi->expects($this->once())
 ->method('fetchApi')
 ->with(123)
 ->willReturn('foo');

 $cachedApi = new CachedApi($innerApi);

 $cachedApi->fetchApi(123);
 $cachedApi->fetchApi(123);
 }

 public function testItCallsInnerForEachId(): void
 {
 $innerApi = $this->createMock(Api::class);
 $innerApi->expects($this->exactly(2))
 ->method('fetchApi')
 ->willReturn('foo');

 $cachedApi = new CachedApi($innerApi);

 $cachedApi->fetchApi(123);
 $cachedApi->fetchApi(456);

 $cachedApi->fetchApi(456);
 $cachedApi->fetchApi(123);
 }
}

If you want to get notified of the next blog post, join the newsletter.

Mastering PHPUnit: Using data providers

Wed, 22 May 2024 00:00:00 +0000

This is part 3 of a series on PHPUnit testing, you can find part 2 here. In this post we’ll focus on the use of data providers. I have written about them before, but for earlier PHPUnit versions. This post is relevant for PHPUnit version 10 or higher. For version 9 or lower check out the original data provider post

What are data providers?

A data provider is a function or method that, as the name suggests, provides data to your tests. So instead of having to write a new test case for every option, we write a single test, and then let it run multiple times with a lot of data, to make sure it handles all cases as expected.

Creating data providers.

A data provider is a static method which provides input data for our tests. This method either lives in your test case itself, or in another class. Generally we want to keep the data provider in the same class, so we keep the data of our test close to the test itself. You’d only move it to another class if the test data is re-used among multiple tests.

For this example we’ll test a ‘calculator’ class which adds two numbers, to show you how that works. So let’s create that class and test case class as discussed in the previous post. And there we add our test method like this:

//Calculator.php

namespace App;

class Calculator
{
 public function add(int $one, int $two): int
 {
 return $one + $two;
 }
}

//CalculatorTest.php

namespace test;

use App\Calculator;
use PHPUnit\Framework\TestCase;

class CalculatorTest extends TestCase
{
 public function testItAdds(): void
 {
 $calculator = new Calculator();

 $this->assertSame(3, $calculator->add(1, 2));
 }
}

Now this is only one case. But we need to deal with negative numbers, zero, etc. We could write out a test case for each of these. But instead, lets create a data provider, to give us this test data.

namespace test;

use App\Calculator;
use Generator;
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

class CalculatorTest extends TestCase
{
 #[DataProvider('provideAddCases')]
 public function testItAdds(int $one, int $two, int $expected): void
 {
 $calculator = new Calculator();

 $this->assertSame($expected, $calculator->add($one, $two));
 }

 public static function provideAddCases(): Generator
 {
 yield [1,2,3];
 }
}

As you can see we added the DataProvider attribute to our test method, and gave it input parameters. The DataProvider attribute takes as an argument the method name on the current test class that we are using. The data from the array that we yield is passed to the test method. So in our example, the dataprovider gives us $one = 1, $two = 2, $expected = 3. (From PHPUnit 11.1 you can also use string keys for the array, which are passed to the argument with the correct name.)

Now if we want to add extra test cases, all we have to do is expand our data provider. So lets add some:

namespace test;

use App\Calculator;
use Generator;
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

class CalculatorTest extends TestCase
{
 #[DataProvider('provideAddCases')]
 public function testItAdds(int $one, int $two, int $expected): void
 {
 $calculator = new Calculator();

 $this->assertSame($expected, $calculator->add($one, $two));
 }

 public static function provideAddCases(): Generator
 {
 yield [1,2,3];
 yield [0,0,0];
 yield [0,3,3];
 yield [-12,7,-5];
 yield [-5,-4,-9];
 yield [-5,0,-5];
 }
}

If we now run our tests, and look at the bottom of the screen we’ll see the following. This means we ran 6 tests, while we only had to write one.

OK (6 tests, 6 assertions)

Why not test in a loop?

Now you may be thinking, why should I use a data provider for this? After all you could just write a foreach loop to go over that data, and run the assertion for each option. The downside of this, is that if one of the checks fail it stops executing right after that. So if our case of yield [-12,7,-5]; failed to work, we would not execute the 2 test cases after that. Meaning we need to fix the first bug, before we can even see if the other test cases are passing. This can be especially annoying when refactoring code, and introducing a bug. Now it becomes harder to spot if you introduced more bugs.

Another upside of these data providers is that you can add multiple data provider to one method. Now for this example it may not be needed, but you may want to split your providers up if you have a lot of test cases. So let’s say that for our calculator test we want to split up the data of negative and positive numbers, we could easily do it like this, and still not have to write a second test.

class CalculatorTest extends TestCase
{
 #[DataProvider('provideAddCases')]
 #[DataProvider('provideNegativeAddCases')]
 public function testItAdds(int $one, int $two, int $expected): void
 {
 $calculator = new Calculator();

 $this->assertSame($expected, $calculator->add($one, $two));
 }

 public static function provideAddCases(): Generator
 {
 yield [1,2,3];
 yield [0,0,0];
 yield [0,3,3];
 }

 public static function provideNegativeAddCases(): Generator
 {
 yield [-12,7,-5];
 yield [-5,-4,-9];
 yield [-5,0,-5];
 }
}

Data providers will also work better if you are tying to test with exceptions. I explain more about that in an earlier blog post about testing exceptions with PHPUnit.

Extra tips for testing with data providers

Data providers are great for quickly finding edge cases of the methods you are testing. So if you are testing something that takes an array, try testing with an empty one, with one value, with two, and maybe 5 or so. If you are testing with strings, try inserting an empty string, or maybe a really large one.

Data providers can also help you with recreating real life scenarios. For example, I had a bug report once of prices on our website displaying weirdly if the price was above 10.0000. Which hadn’t really happened because most products were under 30 bucks, but it does happen in some currencies which are a lot bigger.

Now since this code was already tested, all I had to do was add a case where I added some really large numbers, and the bug was reproduced right away. And because the code was very well tested, the change was easy to make without breaking anything for smaller numbers.

Next weeks post will discuss Mocking and Stubbing in PHPUnit, which will help us deal with dependencies we don’t want to use while running our tests. You can find that post right here.

If you want to get notified of the next blog post, join the newsletter.

Mastering PHPUnit: Writing Your First Test with PHPUnit

Fri, 17 May 2024 00:00:00 +0000

This is part 2 of a series on PHPUnit testing, you can find part 1 here. This post will have us writing our first unit test, and explain some of the basics in doing so.

TestCase class

Every test we write has to extend the PHPUnit\Framework\TestCase class. PHPUnit will look for classes in its configured folders that extend from this class, to execute tests. Within your class it will look for all methods that have the PHPUnit\Framework\Attributes\Test attribute (#[Test]), or where the name of the method starts with test. If either of those is true, PHPUnit will consider it a test, and run it as such.

PHPUnit assertions

As mentioned in the first post of this series, we want to split up our code in three parts, arrange, act and assert. Especially for this third step, the assert, there are a lot of options built right into PHPUnit. We have a lot of assertions out of the box. To name some:

assertSame
assertNotSame
assertEquals
assertTrue
assertFalse
assertNull

And many more.

Generally you want to use the most specific assertions. So instead of doing self::assertSame(true, $myValue), you would want to use self::assertTrue($myValue). If you are using a tool like PHPStan, you can use the phpunit extension to automatically detect when you can use these.

An assert call does what you expect it to, it asserts that something is what you want it to be. With assertTrue you assert your value is true. With assertSame you assert 2 variables are the same, with assertNotSame you assert they are not the same.

What is important to know about these assertions is that they throw a specific PHPUnit exception if they fail, so test execution stops at that point. So if you take the following code example, the self::assertSame(3,4); will not be executed.

self::assertTrue(false);
self::assertSame(3,4);

For a full list of all the possible assertions, you can look at the official documentation

Avoiding code duplication it tests

A lot of the time, your tests will share most of their setup. In case of more legacy projects that need a lot of setup to run, you might want to extract that. To facilitate that, you can override the following methods in your test class:

setUp
tearDown
setUpBeforeClass (static)
tearDownAfterClass (static)

The first method to be called is setUpBeforeClass this is a static method that is called before any test are ran. Next the setUp is called, which is called before every test itself, so if you have 4 tests in your class, setUp will be called 4 times, just before each test is started.

After the test is finished tearDown is called, in the same matter as setUp, it’s called every time a test is finished. And lastly, once all tests fo the class are finished, tearDownAfterClass is called.

If you are familiar with a javascript framework like Jest, the setUp and tearDown are the PHPUnit equivalent of beforeEach and afterEach.

Writing a test with setUp and tearDown

Let’s say we have some code that looks like this. I know it doesn’t look good, but it is common in older applications to have global variables that are manipulated.

$globalVariable = 0;
function globalManipulation (): int {
 global $globalVariable;

 $globalVariable++;

 return $globalVariable;
}

Let’s write some tests for this code. We’ll have one method where we call it once, (and then it should be 1). And then another method where we call it twice,and it should be two.

class TestGlobalFunction extends TestCase
{
 public function testItIsOne()
 {
 $value = globalManipulation();

 $this->assertSame(1, $value);
 }

 public function testItIsTwo()
 {
 globalManipulation();
 $value = globalManipulation();

 $this->assertSame(2, $value);
 }
}

Now if we run the test we get an output which tells us we have a failure.

There was 1 failure:

1) tests\TestGlobalFunction::testItIsTwo
Failed asserting that 3 is identical to 2.

The reason that the value is 3, and not 2, is because we haven’t reset the $globalVariable, so let’s implement setUp and tearDown to reset the variable each time.

class TestGlobalFunction extends TestCase
{
 protected function setUp(): void
 {
 global $globalVariable;
 $globalVariable = 0;
 }

 protected function tearDown(): void
 {
 global $globalVariable;
 $globalVariable = 0;
 }

 public function testItIsOne()
 {
 $value = globalManipulation();

 $this->assertSame(1, $value);
 }

 public function testItIsTwo()
 {
 globalManipulation();
 $value = globalManipulation();

 $this->assertSame(2, $value);
 }
}

Now as you can see we added the logic to both setUp and tearDown. In the setUp we change it so we know our tests start with a predictable state. In the tearDown we set it, so we know our test don’t influence other tests.

In next weeks post we’ll discuss Data Providers in PHPUnit, which will help us write a lot of test cases in a short time, and reduce code duplication. If you want to get notified of the next blog post, join the newsletter.

Mastering PHPUnit: an introduction

Fri, 03 May 2024 00:00:00 +0000

In this blog series we’ll go from writing our first unit test to being a PHPUnit master. This first post will go over the basics, and introduce you to PHPUnit

What is PHPUnit?

PHPUnit is the de facto testing framework for PHP. It initially started in 2001, over 23 years ago at this point. And even today it is still under active development. All its development is done on , where you can open issues if you have a feature request, or find a bug.

Why do we test?

There is a lot of talk about whether you should even write unit tests. To summarize my thoughts on the matter: You should. I don’t trust myself (or anyone else) to write perfect code. Which is why we need to validate that our code works. Now we could manually verify every bit of code we write. For web development we could reload a page, click a few buttons, and consider our code working. But, for every change we make we’ll have to do that work again.

With unit testing we validate this automatically. We write a test that checks that our code does what we think it does. Now instead of having to perform all the possible actions of our code every time we refactor, we simply run our unit tests, and we know in a matter of milliseconds if our code works.

The ‘downside’ of unit testing is that we have to write these tests, which means we have to write more code. It also means that on top of code we need to maintain our unit tests as well. Which means dependency updates, refactors etc. In my opinion the investment in unit tests outweighs the costs.

AAA: The three steps to every unit test

In general, we can split up our tests in three parts: arrange, act, assert.

Arrange

In the arrange phase we do all the set up to make sure that we can run the code that is needed.

We may need to set up some mocks (more about those in a future post), set up the environment or do other things.

Act

In the act phase we run the actual code that we want to execute. In unit testing this is usually just one line of code, where we run the code.

Assert

After we run the code that we want to test, we of course need to check that the code did what we want to do. So we assert that it did what we expected it to do. Here we’ll check either the output of the method test, or check for the side effects that the method caused.

Adding a test to a project

Let’s assume we have a project with the following structure. For this i’ll assume you’ve worked with composer before and have autoloading etc. set up.

📦 project
┣ 📂 src
┃ ┗ 📜 Calculator.php
┣ 📜 composer.json

With the Calculator.php looking like this:

<?php

declare(strict_types=1);

namespace App;

class Calculator
{
 public function add(int $one, int $two): int
 {
 return $one + $two;
 }
}

First off we’ll run composer require --dev phpunit/phpunit, to install phpunit as a dev dependency.

Then we’ll create a file called phpunit.xml to configure phpunit with. (We’ll go into detail about this config file in a future post.)

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
>
 <testsuites>
 <testsuite name="AllTests">
 <directory>test/</directory>
 </testsuite>
 </testsuites>
</phpunit>

Then we’ll continue by creating a file called CalculatorTest.php in the test folder we’ll create.

So now our project will look like this:

📦 project
┣ 📂 src
┃ ┗ 📜 Calculator.php
┣ 📂 test
┃ ┗ 📜 CalculatorTest.php
┣ 📜 composer.json
┣ 📜 phpunit.xml

Now we’ll write our first test. For now just an empty test, which doesn’t do anything.

<?php

declare(strict_types=1);

namespace App\Test;

use App\Calculator;
use PHPUnit\Framework\TestCase;

class CalculatorTest extends TestCase
{
 public function testAdd(): void
 {
 }
}

If we now run phpunit, we’ll get an output like this:

$ vendor/bin/phpunit
PHPUnit 11.1.3 by Sebastian Bergmann and contributors.

Runtime: PHP 8.3.6
Configuration: /Volumes/SourceCode/Private/hugo-blog-test/phpu/phpunit.xml

R 1 / 1 (100%)

Time: 00:00.007, Memory: 6.00 MB

There was 1 risky test:

1) App\Test\CalculatorTest::testAdd
This test did not perform any assertions

/Volumes/SourceCode/Private/hugo-blog-test/phpu/test/CalculatorTest.php:12

OK, but there were issues!
Tests: 1, Assertions: 0, Risky: 1.

PHPunit has executed our test, but it didn’t do any assertions, which is a problem.

So let actually test something, and fill in the testAdd method, like so:

public function testAdd(): void
{
 // arrange
 $calculator = new Calculator();

 //act
 $output = $calculator->add(2, 5);

 //assert
 self::assertSame(7, $output);
}

As you can see we have the three steps, arrange, act, assert. Now in normal development you wouldn’t explicitly add these comments of the different phases, but it’s good to keep in mind how you structure your test.

Now if we run phpunit we’ll get the following output:

$ vendor/bin/phpunit
PHPUnit 11.1.3 by Sebastian Bergmann and contributors.

Runtime: PHP 8.3.6
Configuration: /Volumes/SourceCode/Private/hugo-blog-test/phpu/phpunit.xml

. 1 / 1 (100%)

Time: 00:00.003, Memory: 6.00 MB

OK (1 test, 1 assertion)

That’s a success! We have a test, and we made an assertion about our code, now if we update the calculator class we can run the test again, and validate our code still works as before.

Next week we’ll go over how to determine what to test, how to test it, and some common problems we can face while testing. If you want to get notified of the , join the newsletter.

Never deploy an untagged dependency again

Fri, 26 Apr 2024 00:00:00 +0000

Today we ran into an issue with a build in one of our projects. Composer could no longer install, as a dependency was gone.

When we opened our pipeline logs we saw the following error:

failed to execute git checkout '<ref>' -
- && git reset --hard '<ref>' --

fatal: reference is not a tree: <ref>

It looks like the commit hash is not available in the repository, maybe the
commit was removed from the branch? Run "composer update acme/dependency"
to resolve this.

And in our composer.json we found a dependency like this:

{
 "require": {
 "acme/dependency": "dev-feat-new-things as 4.3.1"
 }
}

Now this worked fine when it got merged, except that after the merge, in the acme/dependency package, the feat-new-things branch got merged into main, so it no longer existed. Therefore, composer stopped being able to find it.

Now while working on a project it’s perfectly fine to put a dependency on a dev branch while you are working on that branch for a new feature, but before releasing it, you want to make sure to actually tag this. Normally we put a comment on a merge request to remind ourselves to tag the release of the package before merging. But this time that was forgotten, and the merge request was merged without a tag of the dependency.

The fix

In order to fix this we wrote the following PHPUnit test.

final class NoDevBranchDependenciesTest extends TestCase
{
 public function testComposerHasNoDevBranchDependencies(): void
 {
 $composer = json_decode(
 file_get_contents(__DIR__ . '/../../composer.json') ?: '',
 true,
 flags: JSON_THROW_ON_ERROR
 );
 $this->assertIsArray($composer);
 $requireDev = $composer['require-dev'] ?? [];
 $this->assertIsArray($requireDev);

 $require = $composer['require'] ?? [];
 $this->assertIsArray($require);

 $branchDependencies = array_filter(
 array_merge($requireDev, $require),
 fn ($version) => str_starts_with($version, 'dev-'),
 );

 $this->assertEmpty(
 $branchDependencies,
 sprintf(
 'Found the following dependencies with a dev key: "%s"',
 implode(', ', array_keys($branchDependencies))
 )
 );
 }
}

Lets go over it step by step and see what it does:

$composer = json_decode(
 file_get_contents(__DIR__ . '/../../composer.json') ?: '',
 true,
 flags: JSON_THROW_ON_ERROR
);
$this->assertIsArray($composer);
$requireDev = $composer['require-dev'] ?? [];
$this->assertIsArray($requireDev);

$require = $composer['require'] ?? [];
$this->assertIsArray($require);

This first bit grabs the contents of the composer.json, and decodes it. The lines after that are grabbing the production and dev dependencies, and does some type checking on this.

$branchDependencies = array_filter(
 array_merge($requireDev, $require),
 fn ($version) => str_starts_with($version, 'dev-'),
);

In the next bit we are looping over all the dependencies, and we only keep the ones that start with dev-, which is used by composer to point to branches.

$this->assertEmpty(
 $branchDependencies,
 sprintf(
 'Found the following dependencies with a dev key: "%s"',
 implode(', ', array_keys($branchDependencies))
 )
);

After we have filtered out the dependencies, all that is left to do is report if we found any. And of course just reporting that an array is not empty to a developer isn’t going to make much sense to them. So we provide a nice message to the other developers to tell them what they did wrong.

By adding this test we make sure that the pipeline stays red until the dependency points to a proper tag. This way it can’t be merged, and we’ll never run into an untagged dependency being merged into main again.

If you want to get notified of the next blog post, join the newsletter.

Stop (ab)using allow failure

Sun, 24 Jul 2022 07:35:43 +0200

We’ve all had our CI fail due to an issue that couldn’t be fixed right away. Usually something like a security issue in a downstream package that can’t be updated yet. This then causes the build to fail, making us unable to merge or deploy. allow_failure seems like a great way to solve this, but it is not.

allow_failure is setting us up for failure. Because now any issue with this check is ignored. And even after the vulnerable dependency is updated, the allow_failure stays. Meaning next time you won’t get an error when there is a vulnerability.

Instead, allow your MR to be merged even when it is red, and allow the deploy step to be executed, even if previous steps fail. This should be discussed with the team beforehand, as we don’t want to merge and deploy when a unit test is failing, or if another check is broken. You only want to do this when you simply can’t fix the issue right away.

In gitlab CI you can use the needs keyword to specify what steps are exactly needed for in order to run the current step. So you can do something like this. Where the 2 build steps have no needs, and the deploy steps only require the build steps. Meaning that even if the other steps fail, you can still run these manual steps.

tests:
 #checks

security:
 #checks

php-build-release:
 stage: build
 needs: []
 # build

nodejs-build-release:
 stage: build
 needs: []
 # build

deploy-test-release:
 stage: deploy
 when: manual
 needs:
 - php-build-release
 - nodejs-build-release
 # deploy

deploy-prod-release:
 stage: deploy
 when: manual
 needs:
 - php-build-release
 - nodejs-build-release
 # deploy

So stop using allow_failure, as it will stop you from seeing new errors popping up. Instead, allow your pipeline to be red for a short while. But do make sure your team can handle this responsibility.

Creating better services

Sat, 13 Nov 2021 00:00:00 +0000

When writing code, you generally want to split up the logic in to different classes. You have your controller classes which take a request, and return a response. You write value objects to represent information which is important to your application. You may write commands, command handlers, repositories and more. The most important group of classes you write are probably your services, which hold (most) of the logic of your app.

There are a few rules which will help you write better, more reliable services, so lets go over them.

Services dont need data classes to be created

A service should not need ‘data’ in order to be created. For example, in an e-commerce setting, you may have a service class which deals with the logic surrounding the cart. This class can never have the current cart as a property, or require it in the constructor.

Of course you can still give the cart as an argument to a method, and perform logic on that. But the class itself should not hold the cart.

Requiring the cart would mean you need to do a lot of operations, just to create this class. You would need to know the ‘current’ cart id, the DB would need to be queried etc. This means the class becomes a lot harder to create.

Services dont have mutable state

A service should not have a data class as a property, or any data that can be mutated. For example, if you have a class which may behave differently in a test environment. Then whether or not this is a test environment should be injected in the constructor. You should not have a setter for this behavior.

When you allow the state of the service to be changed, then the service may have different behavior every single time, and it becomes unpredictable what happens.

An exception here is if you have a cache layer in your service. e.g. an array holding previously calculated results.

Services dont depend on a context class

A ‘context’ class is a class that knows about the current context. This may be a Request object, or an InputInterface for the CLI. Other examples are the symfony Security class, which knows about the current user. Instead the data of the current context that this service needs should be passed along with in a parameter of the method.

Doing so will allow you to use the service in a different context. For example a class that requires the InputInterface object is not usable in a web environment. And if you give the Security class to a service you become unable to ‘use’ another user, for example in the CLI.

Every method should do only 1 thing

This is true for almost all classes, but is worth stating again for services.

Instead of a method doing logging, querying the database and handling logic, let your service do exactly one thing. Move the database stuff to a repository, and handle the logging with a Decorator.

By splitting up the logic, your methods become smaller, easier to reason about, and easier to test.

Differential serving with webpack encore

Sat, 19 Jun 2021 00:00:00 +0000

Recently we’ve been looking into differential serving. With our set up there were a few complications, so in this post i’ll share how we overcame those, and how to set up differential serving with webpack encore.

If you want an explanation of what differential serving is, or how to do with with a normal webpack setup, i suggest reading this post.

In our project we use webpack encore. We use a babel.config.json to manage our babel settings, as jest can’t read that info from the webpack encore file. We also have a .browserslistrc file, in which we configure what browsers we support.

In order to make this work with webpack encore we need to do a few things. First we need to split up our webpack into 3 configs. We need our ‘base’ config. I’ll call this webpack.base.config.js. This contains all our webpack configuration, except for setOutputPath and setPublicPath. We’ll export the Encore object, instead of the getWebpackConfig. So rename your webpack.config.js and remove those lines.

I’ve taken the symfony demo project as an example, so the webpack.base.config.js will look like this:

const Encore = require('@symfony/webpack-encore');
Encore
 .addEntry('app', './assets/app.js')
 .addEntry('login', './assets/login.js')
 .addEntry('admin', './assets/admin.js')
 .addEntry('search', './assets/search.js')
 .enableStimulusBridge('./assets/controllers.json')
 .splitEntryChunks()
 .enableSingleRuntimeChunk()
 .cleanupOutputBeforeBuild()
 .enableBuildNotifications()
 .enableSourceMaps(!Encore.isProduction())
 .enableVersioning(Encore.isProduction())
 .enableSassLoader()
 .enableIntegrityHashes(Encore.isProduction())
 .autoProvidejQuery()
 .autoProvideVariables({
 "window.Bloodhound": require.resolve('bloodhound-js'),
 "jQuery.tagsinput": "bootstrap-tagsinput"
 })
;

module.exports = Encore;

Now, we need the ’legacy’ and the ‘modern’ builds which will support all the older browsers. For that we create a webpack.legacy.config.js and a webpack.modern.config.js with the following content. The name is set so we can refer to that later in our config file.

// webpack.legacy.config.js
const Encore = require('./webpack.base.config');

process.env.BROWSERSLIST_ENV = 'legacy';
Encore
 .setOutputPath('public/legacy-build/')
 .setPublicPath('/legacy-build')
;

const config = Encore.getWebpackConfig();
config.name = 'legacy';

module.exports = config;

// webpack.modern.config.js
const Encore = require('./webpack.base.config');

process.env.BROWSERSLIST_ENV = 'modern';
Encore
 .setOutputPath('public/modern-build/')
 .setPublicPath('/modern-build')
;

const config = Encore.getWebpackConfig();
config.name = 'modern';

module.exports = config;

The key here is the BROWSERSLIST_ENV that is being set. As now we’ll edit our .browserslistrc file. Instead of having the config in a top level in this file, we have 2 versions, the legacy and the modern. By setting the BROWSERSLIST_ENV environment variable to either, we tell browserlist which one to use. This is also the reason we need to have these 2 in separate files. Otherwise the env variables will override each other.

# .browserslistrc
[legacy]
Last 2 versions
IE >= 11
not IE 10
not ie_mob 10

[modern]
chrome >= 80
firefox >= 80
safari >= 12

Now we’ll need to edit our package.json file to build both at the same time. So we change our scripts part to the following:

"scripts": {
 "dev-server": "encore dev-server -c webpack.modern.config.js & encore dev-server -c webpack.legacy.config.js",
 "dev": "encore dev -c webpack.modern.config.js & encore dev -c webpack.legacy.config.js & wait",
 "watch": "encore dev --watch -c webpack.modern.config.js & encore dev --watch -c webpack.legacy.config.js",
 "build": "encore production --progress -c webpack.modern.config.js & encore production --progress -c webpack.legacy.config.js & wait"
}

By using & and wait we run both processes at the same time, instead of one by one. You can also use npm-run-all or something similar.

Next up we’ll change our config/packages/webpack_encore.yaml to tell it about these two builds. Now we can refer to those builds when rendering our tags.

webpack_encore:
 output_path: false
 builds:
 legacy: '%kernel.project_dir%/public/legacy-build'
 modern: '%kernel.project_dir%/public/modern-build'

However, it may be cumbersome to remember to call encore_entry_script_tags twice. Once for legacy, and once for modern. And then also remember to give it the correct attributes. (nomodule and type=module). So instead lets create some twig functions to always render both.

Now, instead of calling encore_entry_script_tags we call script_tags, which will render both script tags. And after we’ve replaced the encore_entry_script_tags, we have differential serving.

<?php
namespace App\Twig;

use Symfony\WebpackEncoreBundle\Asset\TagRenderer;
use Twig\Extension\AbstractExtension;
use Twig\TwigFunction;

final class EncoreExtension extends AbstractExtension
{
 public function __construct(
 private TagRenderer $tagRenderer
 ) {}

 public function getFunctions(): array
 {
 return [
 new TwigFunction('script_tags', [$this, 'legacyEntry'], ['is_safe' => ['html']]),
 new TwigFunction('link_tags', [$this, 'linkEntry'], ['is_safe' => ['html']]),
 ];
 }

 public function legacyEntry($name): string
 {
 return $this->tagRenderer->renderWebpackScriptTags($name, null, 'legacy', ['nomodule' => null]) .
 $this->tagRenderer->renderWebpackScriptTags($name, null, 'modern', ['type' => 'module']);
 }

 public function linkEntry(string $name): string
 {
 return $this->tagRenderer->renderWebpackLinkTags($name, null, 'legacy');
 }
}

If you want to see all the changes, you can check this commit

PHP needs an "unknown" type

Fri, 21 May 2021 00:00:00 +0000

If you are familiar with Typescript, you may know their any type. With any you can do anything. It signals that you don’t know about the actual type, and that anything goes. The following code is completely valid in Typescript.

const doTheThing = (input: any) => {
 input + input;
 input.toString();

 Object.keys(input).map((key) => {
 return input.foo[key];
 });

 const [one, two] = input;
 input();
}

In Typescript 3.0 the unknown type was introduced. unknown is both like any, and completely different. unknown also means that it can be any value, but the logic about it is inverted. With any, anything is allowed, because we don’t know what it is. With unknown, nothing is allowed, because we don’t know what it is.

const doTheThing = (input: unknown) => {
 input + input; // error
 input.toString(); // error

 Object.keys(input).map((key) => { // error
 return input.foo[key]; // error
 });

 const [one, two] = input; // error
 input(); // error
}

In PHP we have don’t have any or unknown, instead we have mixed. Which is an official type since PHP 8.0. And when using static analysis tools the results are mixed.

If we take the following piece of code, and check it with PHPStan, we get no errors. With Psalm we don’t get any errors on level 2, but on level 1 we do.

function doTheThing(mixed $input): void {
 $input->toString();
 $keys = array_keys($input);
 [$one, $two] = $input;
 $input();
}

If PHP would have an unknown type, then it could have the same behavior with both tools. Making changes to mixed would be a big BC break, especially for legacy projects where there is a lot of that. Unknown wouldn’t be a type that should be in PHP itself, but it could live as an annotation.

If we have unknown, we can allow users to use the feature on lower strictness levels. It would also prevent them from having to add potentially thousands of ignores to a baseline file.

Using CSS modules in react when testing with enzyme

Sat, 24 Apr 2021 00:00:00 +0000

We’ve been using CSS modules in a react project, and wanted to use those classes in our tests. However we quickly ran into the problem that none of the classes were found. Any wrapper.exists would be false. The problem was with our css modules, as they were not imported correctly during tests.

Our code looked like this:

import styles from './QA.scss?module';

interface QAProps {
 question: string;
 answer?: string;
 isOpen?: boolean;
}

const QA:React.FC<QAProps> = ({question, answer, isOpen}) => (
 <div>
 <p>{question}</p>
 {isOpen && <div className={styles.Answer}>{answer}</div>}
 </div>
);

When testing it we tried the following:

it('has no answer when not opened', () => {
 const wrapper = shallow(<QA question="hi" />)
 // passes
 expect(wrapper.exists('.Answer')).toBe(false);
});

it('has an answer when opened', () => {
 const wrapper = shallow(<QA question="hi" answer="answer" isOpen />)
 // fails
 expect(wrapper.exists('.Answer')).toBe(true);
});

The problem lied in our fileMock for jest.

 // jest.config.js
 "moduleNameMapper": {
 "\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$": "<rootDir>/__mocks__/fileMock.js",
 "\\.(scss|less|scss\\?module)$": "<rootDir>/__mocks__/fileMock.js"
 },

// __mocks__/fileMock.js
module.exports = 'content';

Because of this export, styles.Answer would be undefined. Instead, during our tests we want it to just return the name of the property. In PHP you would use the __get magic method. (Un)fortunately, this is also possible in js, through a Proxy object.

We make a proxy of an empty object, that always returns the property that is being requested. And in the jest.config.js we update the css files to refer to this new cssMock.js file.

// __mocks__/cssMock.js
const target = {};

const handler = {
 get: function(target, prop) {
 return prop;
 }
};

const proxy = new Proxy(target, handler);

export default proxy;

Now, our tests pass, and our css modules can be used within our tests.

Testing Exceptions in PHPUnit

Fri, 09 Apr 2021 00:00:00 +0000

Last week i gave 10 phpunit tips. This week we’ll take a look at testing exceptions, which wasn’t covered in that post.

Lets start with some example code that we will be testing. We have the Email and EmailValidator classes. Email is a value object that makes sure it is a valid email. We use the EmailValidator to make sure that the emails are only from our company.

//Email.php
final class Email
{
 private function __construct(
 private string $email
 ){}

 public static function create(string $email): self
 {
 if (! filter_var($email, FILTER_VALIDATE_EMAIL)) {
 throw new InvalidArgumentException(sprintf(
 '"%s" is not a valid email',
 $email
 ));
 }

 return new self($email);
 }

 public function asString(): string
 {
 return $this->email;
 }
}

//EmailValidator.php
final class EmailValidator
{
 public function validateCompanyEmail(Email $email): void
 {
 if (! str_ends_with($email->asString(), '@company.com')) {
 throw new InvalidArgumentException(
 'Only "@company.com" emails are allowed'
 );
 }
 }
}

Now, we create a test that makes sure we don’t allow emails other than @company.com. We use the expectException method to tell PHPUnit that we expect this exception. If it is not thrown, or if another exception is thrown, then this test will fail.

final class EmailValidatorTest extends TestCase
{
 public function testItValidatesEmail(): void
 {
 $this->expectException(InvalidArgumentException::class);

 $email = Email::create('foo@bar.com');

 $validator = new EmailValidator();
 $validator->validateCompanyEmail($email);
 }
}

However, we made an error in this test. By starting with the expectException, the test will pass as long the InvalidArgumentException is thrown anywhere within this test. So if we make a typo, and pass foo@barcom to the Email::create, an exception will be thrown. And the test will pass. But then we never test the EmailValidator.

To make sure we are properly testing the exception we need to call expectException just before the method that throws. Now if we run the test, it still passes. But if we make a mistake, and Email::create throws an exception, our test will fail.

final class EmailValidatorTest extends TestCase
{
 public function testItValidatesEmail(): void
 {
 $email = Email::create('foo@bar.com');

 $validator = new EmailValidator();

 $this->expectException(InvalidArgumentException::class);
 $validator->validateCompanyEmail($email);
 }
}

Testing multiple exception paths

Your method may throw exceptions for different reasons. For example, lets say our email validator also makes sure that our colleague bob can’t log in. So the method would look like this.

public function validateCompanyEmail(Email $email): void
{
 if (! str_ends_with($email->asString(), '@company.com')) {
 throw new InvalidArgumentException(
 'Only "@company.com" emails are allowed'
 );
 }

 if ($email->asString() === 'bob@company.com') {
 throw new InvalidArgumentException(
 'bob is no longer allowed to log in'
 );
 }
}

Now we need to test the second path. So lets write that.

public function testBobCantLogIn(): void
{
 $email = Email::create('bob@bar.com');

 $validator = new EmailValidator();

 $this->expectException(InvalidArgumentException::class);
 $validator->validateCompanyEmail($email);
}

But we forgot to turn bar.com into company.com. The test is still green, but we don’t actually test that bob can’t log in.

There are a few options here. We could introduce different exception classes for each error state. Or we can use expectExceptionMessage to make sure we get the correct error. So if we add $this->expectExceptionMessage('bob is no longer allowed to log in'); just after the expectException call, then our test turns red. If we then change @bar.com to @company.com, our test turns green again.

The best practice would be to do both. Introducing new exception classes per error means we can more easily catch the right error. However, you may be dealing with legacy code, where a new exception class could break things. Validating you get the right error message is really important if that message faces users. But even if it is only for other devs, you want to make sure you get the correct exception.

Testing Exceptions with data

Sometimes we use exceptions to pass data back. Perhaps a validator that can have multiple failures, and that has an array of all the errors. Lets take a look at the FormValidator class, and how we would test it. This class validates some information, and may throw an exception, containing the data of all things that went wrong.

final class FormValidator
{
 public function validate(
 DateTimeImmutable $start,
 DateTimeImmutable $end,
 int $newId,
 string $description
 ): void {
 $errors = [];
 if ($start > $end) {
 $errors[] = 'End must be after start';
 }

 if ($newId <= 0) {
 $errors[] = 'The new id must be greater than 0';
 }

 if ($description === '') {
 $errors[] = 'Description can not be empty';
 }

 if ($errors !== []) {
 throw FormValidationException::fromValidationErrors($errors);
 }
 }
}

If we need to test this, using expectException doesn’t make much sense. expectExceptionMessage wont help us out either, as it doesn’t have a message. Instead we can use the good only try catch here.

final class FormValidatorTest extends TestCase
{
 public function testValidatesMultipleErrors(): void
 {
 $validator = new FormValidator();

 try {
 $validator->validate(
 new DateTimeImmutable('2020-01-01'),
 new DateTimeImmutable('1999-01-01'),
 -3,
 ''
 );
 $this->fail('FormValidationException was not thrown');
 } catch (FormValidationException $e) {
 $this->assertSame(
 [
 'End must be after start',
 'The new id must be greater than 0',
 'Description can not be empty',
 ],
 $e->getErrors()
 );
 }
 }
}

Important here is the fail after the validation. If the method throws no exception, we need to fail the test. If we don’t add that call then our test would pass if no exception was thrown.

But, if you don’t need to check any details, other than the exception class, message or code, use expectException.

In conclusion

You can use expectException to test your exceptions. But only set the exception just before it is thrown. Use expectExceptionMessage if the message is important, or if it is the only way to see where something went wrong. Use try catch if you need to validate specific properties of the exception.

For this post, PHPUnit 9.5 and PHP 8.0 were used. If you want to run the tests for yourself, you can find them on github. If you were looking for information on how to test warnings and notices, i have a post about that as well right here.

If you want to get notified of the next blog post, join the newsletter.

10 PHPUnit Tips

Fri, 02 Apr 2021 00:00:00 +0000

PHPUnit is the defacto testing framework for PHP. In this post i want to share with you my top 10 tips for PHPUnit. I’m using PHPUnit 9.5, but most of these apply to older versions as well. So, lets get this party started.

1: Stop using `assertEquals`

One of the most common mistakes is using assertEquals. Instead, you should be using assertSame. When using equals we are doing an == check, instead of ===. Which means that the following test passes.

public function testThatPasses(): void
{
 $this->assertEquals('3', 3);
}

However, if we use assertSame, we are doing === checks, which also take types into account. Meaning the values have to be the same value, and the same type. So the following test would fail, with the following error message: Failed asserting that 3 is identical to '3'.

public function testThatFails(): void
{
 $this->assertSame('3', 3);
}

2: Use a data provider when possible

When you need to test a lot of variations of a test, using a data provider will save you from copy pasting the same set up code everywhere. I wrote a post about this already, so check that out right here.

3: Use `createStub` instead of `createMock`

Since PHPUnit 8.4 we have createStub. Which solved one of the issues with mocking in PHPUnit. Before that, everything was a mock. While in general testing terms there are mocks, stubs and spies.

A mock object is an object where you configure expectations on. So you can configure that a method must be called exactly once, and you can configure with what params the method should be called.

A stub on the other hand is an object that is just a placeholder. This can mean that it isn’t actually used in the part you are testing. Or it means that you only configure that a method will return certain values when called. The difference here is that you don’t test how many times a method is called, or with what params.

By using createStub you starting using the correct terminology for your test doubles. This should help other better understand what is going on in your test suite.

4: Inject static classes

Not too long ago we had to test a method that used an external SDK, which was using a lot of global state. A simplified version of our method looked like this:

function sendRequest(array $data): bool
{
 $result = Transaction::start($data);

 return (bool) $result['body']['data']['success'];
}

Transaction::start could throw a specific exception, and we needed to handle that. But we can’t mock transaction like this, so we needed to change this method to make it testable. We changed the method to the following, to allow us to inject transaction, for testing only

function sendRequest(array $data, ?Transaction $transaction = null): bool
{
 $transaction ??= new Transcation();
 $result = $transaction::start($data);

 return (bool) $result['body']['data']['success'];
}

Now, in our test we can inject the Transaction class.

public function testSendRequest(): void
{
 $transaction = new class extends Transaction
 {
 public static function start(array $data)
 {
 throw new ApiException();
 }
 };

 $this->assertFalse(sendRequest([], $transaction));
}

This test now fails, as we get the exception instead of returning false on this exception. But now we can modify our code to make the test pass.

5: Make your tests fail

You want to make sure your tests are doing their job. So you want to be sure that they are actually testing what you think they are. So lets take a look at the following basic validation function. It checks that the two inputs are both integers.

function validateInput($one, $two): void
{
 if (!is_int($one)) {
 throw new InvalidArgumentException('one must be an int');
 }

 if (!is_int($two)) {
 throw new InvalidArgumentException('two must be an int');
 }
}

We could then create a test for this method like so:

public function testBothMustBeIntegers(): void
{
 $this->expectException(InvalidArgumentException::class);
 validateInput('1', '2');
}

But, if we remove the first (or the second) check, the test still passes. This test isn’t strict enough. Instead, we need more test cases.

function validateInput($one, $two): void
{
- if (!is_int($one)) {
- throw new InvalidArgumentException('one must be an int');
- }

 if (!is_int($two)) {
 throw new InvalidArgumentException('two must be an int');
 }
}

/**
 * @dataprovider provideNonIntegersCases
 */
public function testBothMustBeIntegers($one, $two): void
{
 $this->expectException(InvalidArgumentException::class);
 validateInput($one, $two);
}

public function provideNonIntegersCases(): Generator
{
 yield ['1', '2'];
 yield [2, '2'];
 yield ['1', 4];
 yield [2, 3.5];
}

Now, we use a data provider to test different cases. And if we remove one of the checks now, a test will fail.

We can do this manually, or we can use a mutation testing framework, like infection to help us with ths.

6: Use static analysis on your tests.

You should be using static analysis on your source code. But you should also use it on your tests. It will help you catch bugs in your tests, just like it does in normal code.

But, it will also make sure you aren’t testing too much. For example if you have the following function that sends an api request.

/**
 * @param string|resource $data
 */
function sendApiRequest($data): void
{
 $this->client->post([
 'body' => $data
 ]);
}

This method takes either a string, or a resource, and uses that as the data to send. If, in your tests you pass a class to this method, just to see what happens, that would give you an error in your static analysis tool. So, since you won’t be able to do that in your normal code, you don’t have to worry about it in your tests.

An important thing to note here, is that you should be running your test suite on the same level as your ’normal’ code. If your test suite is stricter, then the error you get there may not show up in your normal code. Ideally both the source and test folder should be checked in the same process.

7: Test the simple stuff

To people who don’t like tests, they always fall in one of two categories. The code is either impossible to test, or it is so simple that it is not worth testing. I’d say, write the simple tests. If the code is worth writing, its worth testing. Generally the time you spend on writing the simple tests isn’t what is going to slow you down.

Even a test like this is worth writing. You wouldn’t be the first (or the last) to accidentally assign the wrong property in the constructor, or read the wrong property in a getter.

Especially if it is a large pull request, these are the kind of thing that get looked over.

public function testItCanBeCreated(): void
{
 $link = new Link('name', '/url');

 $this->assertSame('name', $link->getName());
 $this->assertSame('/url', $link->getUrl());
}

8: Do a sanity check

Be kind to yourself, and do a sanity check before test. Especially something that interacts with a global state of any kind. If you do a quick check before hand, you make sure your test is still doing what it is supposed to do.

For example, we wanted to make sure our SessionFactory did not start a php session when creating the session object. Here we check that we do not have a session id, before we ever start the test. I always add a sanity check comment, so that everyone understands this is not what is actually tested, but a prerequisite.

/**
 * @runInSeparateProcess
 */
public function testNewSessionDoesNotStartAPhpSession(): void
{
 // sanity check
 $this->assertSame(
 '',
 session_id(),
 'A session was already started before this test ran.'
 );

 $factory = new SessionFactory();
 $session = $factory->newSession();

 $this->assertSame('', session_id());
 $this->assertFalse($session->isStarted());
}

9: Test 3 (or more) variants of your loops

Whenever you have a loop in your code, whether it’s a for, foreach, while or do while loop. Try to test at least 3 versions of it.

No loops
1 loop
3 loops

This depends on your code though. If the loop is never reached if the input array is empty, then that it not something you have to consider. But by testing these 3 versions you account for a few potential bugs.

You may be using variables form inside the loop after it. Since these variables don’t exists if you never loop, this should error in your tests. Or when a variable is reused in the next loop by accident, for example if it is declared within an if. When you do multiple loops, this should give you a wrong result if you do multiple loops.

10: Just start writing the tests

A mediocre test is better then no tests. You may write test that aren’t perfect. But when you start writing code it’s generally not perfect either. The only real way to improve the tests you write is by writing more of them.

If you want to get notified of the next blog post, join the newsletter.

Help my Doctrine migrations are broken!

Thu, 25 Mar 2021 21:16:07 +0100

Recently we ran into a problem with doctrine migrations. The schema was manually edited, and wrong migrations were committed to the repository. For a dev (and even staging) env you could just drop the database, fix the migrations, and run them again. But we didn’t want to lose our production data. So how do we fix this?

We really had 2 problems. Our migrations were wrong, and the (production) database was not in the correct state.

Fixing the migrations

The migrations looked (something like) this:

Migration 1: Create users table
Migration 2: Create products table. Create category table
Migration 3: Create category table. Alter users table

So migration 2 and 3 had some of the same content. The fix here was to just remove the creation of the category table from migration 3. Normally you shouldn’t edit migrations, but this was the only way to keep them working. We tested the migrations in a local environment, and were able to run them starting from an empty DB. The down migrations were also working.

This was step one, as now anyone can work on the project and get their DB in the correct state, without manual extra work. However there was another issue, the production database.

Fixing the production database

So now we were ready to take a look at the production database. It turned out the database was in the ‘correct’ state. However the migrations themselves were only ran up until migration 2. Migration 3 was never ran, but the altering of the users table had been done. Thankfully, doctrine has a command for this. The version command will add (or delete) a migration from your migration table, without executing it. In symfony this is exposed as bin/console doctrine:migrations:version.

So to manually add the migration, we ran the following command:

$ bin/console doctrine:migrations:version DoctrineMigrations\\Version20210125124213 --add

DoctrineMigrations\\Version20210125124213 Is the FQN of the migration. The --add is to add the migration. If we needed to remove one we could’ve used --delete. Using this command is safer than manually editing the migrations table, because if you make a typo in the name, doctrine wont add it.

Lessons learned

There are a few lessons to learn here. The first one is that if you mess up your migrations and your database, there is a way to fix it. However, its even better to prevent these issues from happening. There are a few precautions you can take.

Test your migrations

Make it a step in your CI to run your migrations, from an empty DB to the latest version, and from the last version all the way back. This will stop you from committing faulty migrations, and can be as simple as the following steps:

script:
 - bin/console doctrine:migrations:migrate --no-interaction # migrations up
 - bin/console doctrine:schema:validate # check that the entities and the schema match
 - bin/console doctrine:migrations:migrate first --no-interaction # migrations down

Don’t manually change the DB

We wouldn’t have gotten into this trouble if we had only used the migrations to update the DB. A developer had ran php bin/console doctrine:schema:update --force to get the DB into the latest state. This (and manual ALTER ) queries shouldn’t be used if you want your DB to be managed by migrations.

Closing words

In reality this was a small and new project. If we had wiped the DB and recreated it, the data would’ve been added back within 5 minutes. But now we have the experience and knowledge on how to deal with these issues. When life gives you a safe training opportunity, take it!

If you want to get notified of the next blog post, join the newsletter.

A problem with Twig

Thu, 18 Mar 2021 18:31:13 +0100

I’m a fan of twig, and wouldn’t consider moving back to plain php. But, it does come with a few problems. In this post we’ll explore one of the problems i have with twig, and how to work around it.

The problem

When you use a twig file, you do not know what variables it needs, what variables i can use, and what types those variables should be. You have to read the template, or execute it and see what errors you get.

This isn’t bad if you’ve worked on a project for a long time. But when you work on a project that isn’t yours, it can be difficult to understand what a template needs. Is that variable passed to the template, or is it a global? What type is it supposed to be? We’ve had issues multiple times where a variable wasn’t passed to a template, and then the site went down.

You can check this by having e2e tests that check all possible paths, and verify your twig templates. But most code bases won’t have those, or the tests take too long to be of value. What if we could be sure what our templates need, and make sure it gets exactly that?

A solution

At first i came up with the following solution. A service per template that would require the correct variables to be passed. This way, all you need to do is find the correct service for the template, and you know what variables it needs. The problem here is that you will need a service for every single template. This may be okay for small apps, but for bigger apps this seems like a bad solution.

<?php
class AccountPageAction
{
 public function __construct(
 private AccountPageView $view,
 private Security $security
 ) {}

 public function __invoke(): Response
 {
 // Deal with things
 return new Response(
 $this->view->render($this->security->getUser())
 );
 }
}

class AccountPageView
{
 public function __construct(
 private Environment $twig
 ) {}

 public function render(User $user) {
 return $this->twig->render('account/view.html.twig', [
 'user' => $user,
 ]);
 }
}

I posted about my ideas on twitter and got some responses. The response from @azjezz seemed like the perfect solution.

One thing that i liked about this approach is also the Renderer class that was used. He gave the example of a Responder class that he used later in that thread.

So after a few changes i ended up with the following Responder and template:

<?php
final class Responder
{
 /**
 * Render the given twig template and return an HTML response.
 *
 * @param array<string, string|list<string>> $headers
 *
 * @throws TwigError
 */
 public function render(Template $template, int $status = 200, array $headers = []): Response
 {
 $content = $this->twig->render($template->getTemplate(), $template->getContext());
 $response = new Response($content, $status, $headers);
 if (!$response->headers->has('Content-Type')) {
 $response->headers->set('Content-Type', 'text/html; charset=UTF-8');
 }

 return $response;
 }
 // The rest of the class
}

interface Template
{
 public function getTemplate(): string;
 /** @return array<mixed> */
 public function getContext(): array;
}

Now, even when the template is used in multiple end points, its clear what the template needs. Passing in the wrong type will alert a tool like PHPStan. The same goes for forgetting to pass in a certain variable. We can even use a tool like deptrac to make sure all our controllers use the responder, instead of rendering twig templates directly.

If your template has a lot of optional variables, you can consider using named arguments. This will save you from having to pass in a lot of null or other default values.

Using this, the original AccountPageAction now looks like this.

<?php
class AccountPageAction
{
 public function __construct(
 private Responder $responder
 ) {}

 public function __invoke(): Response
 {
 // Deal with things
 return $this->responder->render(
 new AccountPageTemplate($this->security->getUser())
 );
 }
}

class AccountPageTemplate implements Template
{
 public function __construct(private User $user) {}

 public function getTemplate(): string
 {
 return 'account/view.html.twig';
 }

 /** @return array<mixed> */
 public function getContext(): array
 {
 return ['user' => $this->user];
 }
}

By using this Template interface and the Responder we now always know what variables our templates need. We will know if we forgot a variable. It can save you a broken website due to a missing variable, and should make your twig a bit safer.

Explore Your Types

Thu, 11 Mar 2021 21:18:16 +0100

Don’t worry, this isn’t a ‘what type of sandwich are you?’ kinda post. Instead we’ll look at how we can safely add types to our legacy code.

Adding types

There are really two ways of adding types. You either declare what you want your types to be, or you declare what the types can be. When writing new code we should always be precise in our types, so we declare what we want. We can say that a property is a string, or that a method only takes a User class as a parameter.

But for our legacy code, this is a bit different. Lets take, for example this snippet from a Login class i recently had to deal with. While password is annotated as a string, when we look at the implementation, we see that false is also an option.

class Login
{
 /**
 * @var string
 */
 private $password;

 public function login($email, $password = false)
 {
 // stuff
 $this->password = $password;
 }
}

Tools like Psalm and PHPStan will tell us there is an error here, or you could find it on your own. This is a problem we’d like to solve, but there are multiple ways of doing so. One would be to add a (string) cast, before the assignment of $password. The problem with that, is that we are changing behaviour of the code. Another place may have a check like $this->password === false.

The best solution would be to annotate password with string|false. And if later it turns out it can also be a resource, pointint to a private key, then we turn it into string|false|resource. Now, if it can also be a Password object, then we make it string|false|resource|Password. We must however, resist just ploping down mixed and calling it a day.

By using mixed we might as well have not added any types. Instead, we want to document our types as precise as possible, wherever we can. Now, the example of the Login class is small, but when you have a class with 200 methods, all calling each other and using its output, finding out what type something is can be difficult. You have to start with a type you are sure of. And if you can, make it an explicit type, in the language.

For example, what if we have the following method in a legacy class. When we read the code, we still don’t know what $orderData can be. Maybe the fetchOrder method will hint us to that. But when we read it we can guarantee that it returns an int or false. We can add it right away, and know we won’t change any behaviour.

class Legacy
{
 // 100 methods

 private function getOrderId($orderData)
 {
 $orderId = $this->database->fetchOrder($orderData);

 if (! $orderId) {
 return false;
 }

 return (int) $orderId;
 }

 // 100 more methods
}

So now we’ve added the type to our method. We may need to update the baseline of our static analysis tool, or maybe we don’t have to do any extra work. Now we can be sure that when we use the getOrderId method, we get either an integer, or false. This (hopefully) means that we can add types elsewhere.

class Legacy
{
 // 100 methods

 private function getOrderId($orderData): int|false
 {
 $orderId = $this->database->fetchOrder($orderData);

 if (! $orderId) {
 return false;
 }

 return (int) $orderId;
 }

 // 100 more methods
}

So whats the point

The first point of adding these types is to increase our understanding of what this code does. By giving it the proper types, we may detect dead code, or find bugs. If we keep up this, and the boyscout rule long enough, then it may not be legacy code anymore.

The second point is that your ’new’ or better code probably still has to connect with legacy code. So by adding a bit more certainty, we are a bit safer that the code does what we expect it to do.

So go out there, and explore all the types your codebase has. So what if the parameter can be one of 12 interfaces, a string, or a boolean. Only once the horrors are documented do you have a chance to improve them.

You should be using PHPStans bleeding edge

Thu, 04 Mar 2021 00:00:00 +0000

Does your project use PHPStan? Then you really should be using the bleeding edge config, regardless of what level you are running. It will make transition to the newer version much easier.

What is bleeding edge

To preserve backwards compatibility as much as possible, new rules, and new settings are delayed until the next major version. For example, using null coalesce on a variable that can not be null will be an error in the next version. But in this version it does not. Bleeding edge gives you the features of the next PHPStan version, right now.

How to activate it

To start using bleeding edge, you need to include it in your phpstan.neon. The next time you run PHPStan, you will get all the new errors.

includes:
 - vendor/phpstan/phpstan/conf/bleedingEdge.neon

Why activate it

Updating composer dependencies should be part of the normal workflow of a team. Whether you take time every monday to do updates, or you update constantly. When using bleeding edge, you have a little bit of work whenever something new gets added. You need to fix the errors, or add them to an ignore. This is only one new rule or setting at a time, meaning you generally don’t have to spend all that much time.

When you wait until the next major version, you suddenly have maybe 10 new rules and settings you need to account for. Meaning you’ll have to spend a far bigger chunk of time then. This could also make it easier to just say “screw it”, and ignore all the new errors. By doing the updates as they come, you keep the tasks small.

You don’t need to be at level max in order to use bleeding edge. If a new rule gets added at level 7, and you are on level 5, that new rule wont activate.

When not to activate it

I would only suggest to do this for a project, with a composer.lock file. If you are creating a library, this could mean that suddenly a nightly build is failing, or that a PR of gets blocked as the new PHPStan errors need to be fixed first.

Even in a library, you could still run a check with bleeding edge, but allow it to fail. That way you still get the feedback that in the next version things will break, but it wont block contributions.

In summary

If your project uses PHPStan, use bleeding edge. It will make your upgrade path smaller, and allows you to use the features of the next version today.

PHPUnit beyond basics: Dataproviders

Fri, 26 Feb 2021 00:00:00 +0000

Once you have set up your first unit tests, and you have a good configuration, its time to add a lot of tests. Lets take a look at using data providers, as a way to test with a lot of data.

For this example we’ll test a piece of code that is supposed to do the following. Given the array ['a', 'b', 'c'], return the string a a-b a-b-c b b-c c. This function has to combine the array values to that string.

Now we could write a test that tests this case. However, we need to make sure this works in more than just this case. It needs to handle duplicate entries, an empty array, an array with just one element, and so on. We could add a test method for every case, but there is an easier solution: dataproviders.

public function testItCombines(): void
{
 $this->assertSame('a a-b a-b-c b b-c c', KeyCombiner::combine(['a', 'b', 'c']));
}

A dataprovider, will provide the input for your tests. Normally a test method does not take any arguments, but with a dataprovider, you can execute it many times, with different data every time. So lets change our test, to use a data provider, for just this case.

/**
 * @dataProvider provideCombineCases
 */
public function testItCombines(array $input, string $expectedOutput): void
{
 $this->assertSame($expectedOutput, KeyCombiner::combine($input));
}

public function provideCombineCases(): Generator
{
 yield [['a', 'b', 'c'], 'a a-b a-b-c b b-c c'];
}

We configure our test method to have a dataprovider by the annotation: @dataProvider provideCombineCases. (If you are using PHPUnit 10, you can use the Datprovider attribute rather than the annotation.). This has to refer to a public method on the same class, which will return an iterable. This iterable has to be an iterable of arrays, where every array matches the arguments of the method it provides data to.

In our case, we have a Generator which provides an array, containing an array, and a string. These match our test method, which takes an array, and a string. Now, adding more test cases, is as simple as adding a new case to the data provider. So lets add the test cases we talked about. And, if we discover a bug, we can add a new test case to the provider, and make sure we cover that too.

public function provideCombineCases(): Generator
{
 yield [['a', 'b', 'c'], 'a a-b a-b-c b b-c c'];
 yield [[], ''];
 yield [['a', 'a', 'b'], 'a a-a a-a-b a a-b b'];
 yield [['b'], 'b'];
}

But, we’re not done here. If we add keys to our dataprovider, we get that information when our tests fail, or when we use the --testdox flag. Currently it will output something along the lines of “Test it combines with dataset 0”.

public function provideCombineCases(): Generator
{
 yield 'multiple keys' => [['a', 'b', 'c'], 'a a-b a-b-c b b-c c'];
 yield 'an empty list' => [[], ''];
 yield 'duplicate keys' => [['a', 'a', 'b'], 'a a-a a-a-b a a-b b'];
 yield 'a single key' => [['b'], 'b'];
}

Now, if we use the testdox flag, we get an output like “Test it combines with multiple keys”. And if we get a failure, the error message will also contain the key you used.

Additional things

Multiple providers

You can use multiple dataproviders on a single method. This can be useful for validation methods. Where you can split up the cases that pass validation, and cases that dont.

/**
 * @dataProvider provideAlnumCases
 * @dataProvider provideNonAlnumCases
 */
public function testItValidatesAlnum(string $input, bool $isAlnum): void
{
 $this->assertSame($isAlnum, Validator::isAlnum($input));
}

public function provideAlnumCases(): Generator
{
 yield ['a', true];
 // more
}

public function provideNonAlnumCases(): Generator
{
 yield ['_', false];
 // more
}

Code coverage

Any code that is executed during the dataprovider does not count toward code coverage. So if your object under test is only constructed in dataproviders the __construct method won’t be considered covered.

Errors

When an Exception (or Error) is thrown in your dataprovider, none of the test cases are executed. Instead it will count as a single (failed) test.

When

Data providers are ran before the setUp or setUpBeforeClass method is called. Meaning you can’t use any properties set in those methods.

PHPUnit beyond basics: configuration

Fri, 19 Feb 2021 00:00:00 +0000

If you just got started with PHPUnit, its configuration file may be a bit daunting. Today we’re gonna walk through (what i consider) the ideal config file. If you’re just here to copy paste the config, then you can find it at thebottom 👇.

A minimum phpunit.xml may look like this:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit>
 <testsuites>
 <testsuite name="Tests">
 <directory>tests/</directory>
 </testsuite>
 </testsuites>
</phpunit>

The first thing we can do to improve this, is link the xsd. By doing so we now get auto completion, and validation of our xml file. If you start typing anywhere, your IDE will help you out.

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
>
 <!--More-->
</phpunit>

PHPUnit comes with a lot of good defaults, but there are a few settings i like to set. Colors is just because i like my colors, but the other settings are important.

<phpunit
 colors= "true"
 executionOrder="random"
 failOnWarning="true"
 failOnRisky="true"
 failOnEmptyTestSuite="true"
 beStrictAboutOutputDuringTests="true"
 verbose= "true"
>
<!--More-->

By using a random execution order, we make sure our tests aren’t depending on each other. For example one test writing to a DB, and the other one is reading from it. This is bad, because then if the first test fails, the next one will too. And it may be unclear why. It also means we can’t just run the second test, as we need to run the other one first.

Fail on waring, and fail on risky will just make your test suite a bit more strict. Note that this setting is not about ’normal’ warnings, but instead warnings from phpunit. There are several reasons why phpunit may consider a test risky, which you can find here. With fail on risky we make sure those tests fail as well.

Fail on empty test suite will make sure your test suite actually contains a test. When you forget an annotation, or have a typo in your test name, then suddenly its not a test anymore, and you code may go untested, without you noticing.

Be strict about output also helps your tests be more strict. Generally, when your tests have output, that is something unintentional. Even if your code under test does an echo, you can catch that with ob_get_clean(), to validate the output of that. So any output that leaks out, should be an error.

Verbose is debatable. It does give you the most information possible on failures and errors, but if you have a test suite with a lot of skipped tests, it may be too verbose. Your best bet is to turn it on, and only if you find the output too much, turn it off.

Now that we have configured the phpunit element, there are only a few things left to do. The next thing we want to do is add some ini configurations.

<phpunit>
 <php>
 <ini name="error_reporting" value="-1"/>
 <ini name="display_errors" value="On"/>
 </php>
 <!--More-->

With error_reporting set to -1, all notices, deprecations, warnings etc, will be reported And with the default configuration, these are converted into exceptions, which mark your test failed.

Setting display_errors to On will probably not be needed for most projects. But if display_errors is off, and log_errors is too, then you will not see any error output. These are on by default in PHP, but if you, or someone who works on your project, has a wonky environment, then you won’t see any output. By forcing it to On, you will still see output of fatal errors.

Last but not least, we’ll set up our coverage:

<phpunit>
 <coverage>
 <include>
 <directory suffix=".php">src/</directory>
 </include>
 </coverage>
</phpunit

We tell phpunit that when it collects coverage, this is where to find it. Note that we don’t set a <logging> element, as that would cause phpunit to always collect coverage if possible. Instead, only pass it when it is needed. We don’t want to always generate coverage, as that will make your tests a lot slower.

Now, when you want to create a code coverage html report, you can do the following:

vendor/bin/phpunit --coverage-html=coverage
cd coverage
php -S localhost:1313

Now you got a nice looking code coverage report on http://localhost:1313.

TLDR

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
 colors= "true"
 executionOrder="random"
 failOnWarning="true"
 failOnRisky="true"
 failOnEmptyTestSuite="true"
 beStrictAboutOutputDuringTests="true"
 verbose= "true"
>
 <php>
 <ini name="display_errors" value="On"/>
 <ini name="error_reporting" value="-1"/>
 </php>

 <testsuites>
 <testsuite name="Tests">
 <directory>tests/</directory>
 </testsuite>
 </testsuites>

 <coverage>
 <include>
 <directory suffix=".php">src/</directory>
 </include>
 </coverage>
</phpunit>

Every day design pattern: Builder

Fri, 12 Feb 2021 19:58:22 +0100

Just like the factory pattern, the builder is a creational pattern, meaning it is about how objects are created. But unlike a factory, a builder allows you to build an object in parts. I tend to use it for creating objects that take a configuration.

Lets start with an example. This class builds a guzzle client, with a certain config. Normally we have a timeout of 10 seconds, with a 5 second timeout for both connect and read.

use GuzzleHttp\Client;
use GuzzleHttp\RequestOptions;

final class GuzzleClientBuilder
{
 private array $config = [
 RequestOptions::TIMEOUT => 10,
 'defaults' => [
 RequestOptions::CONNECT_TIMEOUT => 5,
 RequestOptions::READ_TIMEOUT => 5,
 ],
 ];

 public static function defaultClient(): Client
 {
 return self::init()->build();
 }

 public static function init(): self
 {
 return new self();
 }

 public function withTimeout(int $timeout): self
 {
 $this->config[RequestOptions::TIMEOUT] = $timeout;

 return $this;
 }

 public function withConnectTimeout(int $timeout): self
 {
 $this->config['defaults'][RequestOptions::CONNECT_TIMEOUT] = $timeout;

 return $this;
 }

 public function withReadTimeout(int $timeout): self
 {
 $this->config['defaults'][RequestOptions::READ_TIMEOUT] = $timeout;

 return $this;
 }

 public function build(): Client
 {
 return new Client($this->config);
 }
}

So if we want to use this builder to create a client with a 30 second timeout, we would do the following:

$guzzle = GuzzleClientBuilder::init()->withTimeout(30)->build();

By using the builder we still have all the other config, so the timeout for connection would still be 5 seconds.

Another example you may have used is a Query Builder. For example the Doctrine Query builder. The query builder, as the name suggests, allows you to build a query. One of the strengths of this is that you can have conditionals, like so:

function getUsers(EntityManager $em, bool $onlyActive): array
{
 $qb = $em->createQueryBuilder();

 $qb->select('u')
 ->from('User', 'u')
 if($onlyActive) {
 $qb->where('active = 1')
 }

 return $qb->getQuery()->getArrayResult();
}

Now at runtime you can determine if you want only the active users, or all users. And you don’t need to do any magic like concat strings together so the sql is valid.

Why use it

A strength of the builder pattern is that it allows you to build up an object bit by bit. But, because you do this before you actually create the object, the end result can be immutable. Meaning you wont need setters on the object. Immutable objects are generally easier to debug and reason about.

The builder object itself is mutable by nature, its sole intention is to be dynamically changed. But, the builder object is usually shot lived, while the resulting object may be used in a lot of places.

In conclusion

The builder pattern is a creational pattern, that allows you to build an object dynamically. It can help in making objects immutable. It can also help make the configuration of an object easier to understand, as it can be done in steps.

Every day design pattern: Factory

Mon, 25 Jan 2021 19:58:22 +0100

The previous two chapter of this blog series were about the decorator and the adapter. These are structural design patterns. Meaning they deal with the structure of a system. They can help with simplifying relationships, and moving responsibilities.

The factory is a completely different type of design pattern. Its a creational pattern. A creation pattern is intended to make creation of objects easier. I generally use one when a class needs a multiple parameters, but only a few need to be ‘dynamic’. Lets take a look at an example.

Lets say we have SaaS where multiple companies use our system. They all have their own accounts. So we have an account service, which takes an account repository, but also the id of the current company. When we create the account service, we need to know this id, but we don’t really care about the repository it uses.

interface AccountRepository
{
 public function findByEmail(string $email, int $companyId): ?Account;
}

class AccountService
{
 public function __construct(
 private AccountRepository $repository,
 private int $companyId
 ) {}

 public function getAccountByEmail(string $email): Account
 {
 $account = $this->repository->findByEmail($email, $this->companyId);
 if ($account === null) {
 throw AccountNotFoundException::fromEmail($email);
 }

 return $account;
 }
}

When we use a factory to create this service it would look like this. When creating the object we need to know the company id. It may be a query parameter, or we know it because of the url that is visited. Whatever the case, we only know this at runtime, and can’t do this beforehand in a configuration or something like that.


class AccountServiceFactory
{
 public function __construct(
 private AccountRepository $repository
 ) {}

 public function create(int $companyId): AccountService
 {
 return new AccountService($this->repository, $companyId);
 }
}

Another reason to create a factory, may be to simplify creation. This is especially useful when you provide a package. Lets look at react-php. They provide a static factory class, which is used to create a LoopInterface. Which one you can use is depending on installed extension, and php versions. The class is even called Factory

An excerpt of the class looks like this. When you need a LoopInterface, all you have to do is Factory::create(). It means you don’t have to worry about having the correct extensions ready, just to get started.

final class Factory
{
 public static function create()
 {
 if (function_exists('uv_loop_new')) {
 return new ExtUvLoop();
 } elseif (class_exists('libev\EventLoop', false)) {
 // more elseifs
 }

 return new StreamSelectLoop();
 }
}

When not to use a factory

Sometimes, i see factories being used when they don’t add any ease of use:

An example may be the following factory:

class UserFactory
{
 public function newUser(
 string $name,
 string $email,
 string $passwordHash
 ): User {
 return new User(
 $name,
 $email,
 $passwordHash
 );
 }
}

Here the factory doesn’t make instantiation of the user any easier, instead it just adds another step before the creation of the object.

In conclusion

The factory pattern is used to make creation of objects easier. Either by deciding which one to create, or by splitting up when certain parameters are required.

Every Day Design Pattern: Adapter

Sun, 10 Jan 2021 00:00:00 +0000

This is the second post in a series of design patterns i use (almost) daily. You will find the other posts at the bottom of this article.

On wikipedia, the adapter pattern is described like so:

the adapter pattern is a software design pattern (…) that allows the interface of an existing class to be used as another interface. It is often used to make existing classes work with others without modifying their source code.

The adapter is used to make a class ‘fit’ the interface you wish to use. Which can be helpful when you have multiple implementations of something, or if you want to migrate from one system to the other.

Multiple implementations

A great example of using adapters to have multiple implementations is Flysystem. Flysystem will let you save to different kinds of backends, without having to change your code. There is one interface you talk to, and behind that can be any adapter which saves it to a specific place, like dropbox, aws S3, or your local filesystem.

For that it has the FileSystemAdapter interface. This is the interface every implementation has to adhere to. Then there is a LocalFileSystemAdapter which implements this interface. Or the FtpAdapter, which also does that.

Now, when you want to save to FTP on your production server, and to the local file system on dev, the only thing you’ll need to change is some configuration, to make sure you get the right class. Now when you decide to switch from FTP to google cloud, you use the GoogleCloudStorageAdapter.

Here using adapters for the same interface allows you more flexibility, and makes upgrading easier.

Migrating between systems

For this example i want to take you back to october 2017. During that time i was working a bit on OpenCFP. OpenCFP used a deprecated library for its authentication: sentry. The idea was to replace sentry with sentinel, a new auth package by the same people that made sentry.

This was done in two steps. The first step was done by Dustin Wheeler. In this pull request, all usages of the sentry authentication where ‘hidden’ behind two interfaces. The Authentication and AccountManagement interfaces. This meant that the only places in the code base that knew about sentry were the implementations of those interfaces.

So now there existed the SentryAuthentication and SentryAccountManagement Only these two places knew of anything related to the authentication/account management. By first moving to this new implementation we made the migration much easier. All we now needed to do was create new implementations, based on the new sentinel package, and we were done.

The sentinel upgrade later came in november 2017 in this pull request. In the end it was still 39 files big, but keep in mind that 18 of those were tests, and another 4 were database migrations. This is of course a lot more than just two implementations of the interface. In the end a few more things were needed, to make sure everything kept working.

By using the two adapters, we could switch between sentry and sentinel, without having to change any of the code using it. All of the logic related to the third party code was in the adapter. This also means that if sentinel had to be updated, and some of the logic would change, this only needed to be done in one place.

Every day design pattern: Decorator

Fri, 01 Jan 2021 00:00:00 +0000

This is the first post in a series of design patterns i use (almost) daily. You will find the other posts at the bottom of this article.

The Decorator pattern

On wikipedia, the decorator pattern is described like so:

In object-oriented programming, the decorator pattern is a design pattern that allows behavior to be added to an individual object, dynamically, without affecting the behavior of other objects from the same class.

Personally, i use this pattern to either add functionality to a class from a third party package, or to separate the concerns of a class. The decorator does so by ‘wrapping’ around the original object.

Adding functionality

Lets say i’m sending emails with the symfony Mailer component. You can add a header to an email that tells the email receiver not to send back any automated mails (like an out of office mail).

Since were sending automated mails, we never want to receive those mails. So we want to add that to every single one of our emails. We do this by decorating the MailerInterface. We take the original mailer as a constructor argument, and then we add our own logic.

This is how you’d normally want to use this design pattern. You take the original class in the constructor, and add your own special logic on top of that.

final class DecoratedMailer implements MailerInterface
{
 public function __construct(
 private MailerInterface $inner
 ) {}

 public function send(RawMessage $message, Envelope $envelope = null): void
 {
 // RawMessage doesn't have headers, so we can only do this if we have a `Message`
 if ($message instanceof Message) {
 $message->getHeaders()
 ->addTextHeader('X-Auto-Response-Suppress', 'OOF, DR, RN, NRN, AutoReply');
 }

 $this->inner->send($message, $envelope);
 }
}

Separation of concerns

A decorator can also be used to separate the concerns of a class. Lets say we have the following Api interface, and its implementation. This class does too much, and its hard(er) to maintain or add features to. What if we wanted to not cache certain requests?

interface Api
{
 public function request(string $method, string $url, array $params): string;
}

class MyApi implements Api
{
 public function __construct(
 private Client $guzzle,
 private LoggerInterface $logger,
 private CacheInterface $cache,
 private string $apiKey
 ) {}

 public function request(string $method, string $url, array $params): string
 {
 $cacheKey = $method . $url . http_build_query($params);
 $cachedResult = $this->cache->get($cacheKey);
 if ($cachedResult !== null) {
 return $cachedResult;
 }

 $this->logger->info(sprintf(
 'Requesting %s - %s',
 $method, $url
 ));

 $params[RequestOptions::HEADERS]['api-key'] = $this->apiKey;

 $result = $this->guzzle->request($method, $url, $params);
 $body = (string) $result->getBody();
 $this->logger->info(sprintf(
 'Response: "%s"',
 $body
 ));

 $this->cache->set($cacheKey, $body);

 return $body;
 }
}

Lets split it up into multiple classes, using the decorator pattern. We turn this into three classes, the decorator that cares about cache, the decorator that cares about logging, an the actual class. Now we can easily remove cache for a specific request. Testing also becomes easier. The test for the CachingApi now only needs to check that items are properly added to cache, or fetched form it.

class CachingApi implements Api
{
 public function __construct(
 private Api $inner,
 private CacheInterface $cache
 ) {}

 public function request(string $method, string $url, array $params): string
 {
 $cacheKey = $method . $url . http_build_query($params);
 $cachedResult = $this->cache->get($cacheKey);
 if ($cachedResult !== null) {
 return $cachedResult;
 }

 $result = $this->inner->request(string $method, string $url, array $params);

 $this->cache->set($cacheKey, $result);

 return $result;
}

class LoggingApi implements Api
{
 public function __construct(
 private Api $inner,
 private LoggerInterface $logger
 ) {}

 public function request(string $method, string $url, array $params): string
 {
 $this->logger->info(sprintf(
 'Requesting %s - %s',
 $method, $url
 ));

 $result = $this->inner->request(string $method, string $url, array $params);

 $this->logger->info(sprintf(
 'Response: "%s"',
 $result
 ));

 return $result;

}

class MyApi implements Api
{
 public function __construct(
 private Client $guzzle,
 private string $apiKey
 ) {}

 public function request(string $method, string $url, array $params): string
 {
 $params[RequestOptions::HEADERS]['api-key'] = $this->apiKey;
 $result = $this->guzzle->request($method, $url, $params);

 return(string) $result->getBody();
 }
}

No interface, no problem

The examples so far have used interfaces. Sometimes the class we wish to decorate does not have an interface. In that case we need to extend the original class. You now have the option to either call the parent directly, or inject the class in the constructor. Personally i would go with injecting the class in the constructor, as this allows you to have multiple decorators on top of each other.

The problem

So i lied a little bit, as there can be a problem when using a decorator when you extend a class. And that is with fluid interfaces. A fluid interface is a class that returns itself on (almost) every method call. Like the following Request class.

We could decorate this no problem, and do the following.

class Request
{
 public function setUrl(string $url)
 {
 $this->url = $url;
 return $this;
 }

 public function setMethod(string $method)
 {
 $this->method = $method;
 return $this;
 }
}

class EchoRequest extends Request
{
 public function setUrl(string $url)
 {
 echo $url;
 $this->inner->setUrl($url);
 return $this;
 }

 public function setMethod(string $method)
 {
 echo $method;
 $this->inner->setMethod($method);
 return $this;
 }
}

But, what happens if the request class gets updated, and a the following method gets added. Suddenly when you call setParams on the decorated class, you are working with the non decorated version. So be careful when using the decorator pattern in combination with fluid interfaces.

class Request
{
 //...
 public function setParams(array $params)
 {
 $this->params = $params;
 return $this;
 }
}

The takeaway

The decorator patterns allows you to add additional functionality to a class, or it can help in splitting up the responsibilities of a class. You may want to be careful when decorating an object that has a fluid interface, as you can suddenly use the non decorated class.

Replacing private services with mocks during tests

Fri, 18 Dec 2020 15:49:00 +0100

Recently i wanted to create an E2E test for a project i was working on. However, for some of its data, the test would talk to some remote API, to fetch data. Which i didn’t want to happen in the test.

So i wrote the following code, and assumed everything would work:

$client = self::createClient();
$repo = new MemoryUserRepository();
self::$container->set(RemoteUserRepository::class, $repo);
$client->request(
 'POST',
 '/users/create',
 [],
 [],
 ['CONTENT_TYPE' => 'application/json'],
 json_encode(
 // user data
 )
);

However, i was quickly greeted with the following error. I had tried to replace a private service.

There was 1 error:

1) App\Tests\E2E\CreateUserTest::testCanCreateUsers
Symfony\Component\DependencyInjection\Exception\InvalidArgumentException: The "App\Domain\Repository\RemoteUserRepository" service is private, you cannot replace it.

The … service is private, you cannot replace it.

A quick google told me that i had 3 options, make all my services public, which has its own issues, or to mark this specific service as public, or to add a custom version of the RemoteUserRepository as a service for my test environment.

The last option would’ve been fine, but i wanted to replace it with different mocks, depending on what i needed for that test. Perhaps i wanted to create a test where i made sure that the remote repository was never called.

Making this service public wouldn’t have been that bad. But it felt weird to change my production container, just for my tests. Thankfully, you can create a services.yaml, just for you tests: services_test.yaml. Another upside of that is that it is clear these changes are just for your tests.

So i ended up with the following services_test.yaml:

#config/services_test.yaml
services:
 App\Domain\Repository\RemoteUserRepository:
 public: true

If you want to get notified of the next blog post, join the newsletter.

Testing code that generates warnings

Thu, 21 Nov 2019 20:51:46 +0100

Our code base has a lot of code that looks like this:

<?php

try {
 $this->doScaryThing();
} catch(Exception $e) {
 trigger_error("Downgraded: " . get_class($e) . ":" . $e->getMessage(), E_USER_WARNING);
}

Or sometimes trigger_error is used as a way to log other thigns. This makes it rather difficult to test. Thankfully PHPUnit 8.4 has the expectWarning method, that allows us to check this:

<?php

public function testItTriggersWarning(): void
{
 $object = new Danger();
 $this->expectWarning();
 $object->doWarningThing();
}

This does mean that the execution stops after the warning is triggered, so we can’t assert anything after that. Thankfully we can still do it in a bit of a ‘hackish’ way:

<?php

public function testItHasCorrectValueAfterWarning(): void
{
 $object = new Danger()
 $ret = @$object->doWarningThing();
 $this->assertSame('good_value', $ret);
}

Personally i like to add an @depends on that second test, so that it requires the test that it triggers a warning to pass before it is executed. Of course this does mean we need two tests per case, one for the triggering of the warning/notice, and one where it is silenced so we can check the output. But it allows us to test the code, and maybe move to a better logging situation in the future.

What is the boy scout rule

Mon, 11 Nov 2019 12:40:49 +0100

The boy scouts have a rule:

Always leave the campsite better than you found it

This means they throw away the trash they see, they clean up the area, and at the very least, they don’t make it any worse.

We can apply this same rule to our code base. Always leave the code cleaner or better than you found it. Over a longer period of time, consistently applying this rule can have great impact on our code base.

But what does the boy scout rule mean in the context of our code? For a camp site its simple to see what can be done. For our code this may prove a bit more difficult. Just recently i spoke with someone who said they couldn’t apply the boy scout rule to their code, as they don’t have the time for it. When investigating a bit further, it turned out they mis understood the boy scout rule. To them it meant making every piece of code you touch perfect. And all the code that it interacts with as well. This is for many code bases close to impossible, and its also not what the boy scout rule is.

The boy scout rule is about making small, continuous improvements to your code. If you are fixing a bug, and find the code around it is a mess, fix a bit of that mess. You don’t have to make it perfect, you don’t have to fix it all, but improve it just a little bit.

Lets look at an example. Recently i found a factory class that was instantiating an object with more parameters than it needed. (Which is totally valid and not a problem for PHP) So i decided to fix that issue.

The starting code looked something like this:

<?php
class ImplementationFactory
{
 private $container;

 public function __construct(Container $container)
 {
 $this->container = $container;
 }

 /**
 * @return ImplementationInterface
 */
 public function createImplementation()
 {
 $config = $this->container->get(Config::class);
 $router = $this->container->get(Router::class);

 if ($config->getEnv() === 'development') {
 return new DevImplementation($config, $router);
 } else {
 return new Implementation($config, $router);
 }
 }
}

There are a few things wrong with this code. The one i was initially going to fix was the fact that these constructors no longer took arguments. But there are more:

The factory takes the complete container as an argument.
We are using an else when it is not needed.
We are checking the env against a string, where a constant may be more appropriate

And after all that, its also the question of whether we should have a different dev implementation for this, and whether that problem should be solved in this factory.

Lets first of all start with the task i was supposed to do:

 public function createImplementation()
 {
 $config = $this->container->get(Config::class);
- $router = $this->container->get(Router::class);

 if ($config->getEnv() === 'development') {
- return new DevImplementation($config, $router);
+ return new DevImplementation();
 } else {
- return new Implementation($config, $router);
+ return new Implementation();
 }
 }

At this point i could close the ticket, and move on. But i apply the boy scout rule I got three points that are wrong with the code. Lets fix the first one: “The factory takes the complete container as an argument.” This isn’t necessarily wrong, but it is far more than what this factory needs to know:

class ImplementationFactory
{
- private $container;
+ private $config

- public function __construct(Container $container)
+ public function __construct(Config $config)
 {
- $this->container = $container;
+ $this->config = $config;
 }

 /**
 * @return ImplementationInterface
 */
 public function createImplementation()
 {
- $config = $this->container->get(Config::class);

- if ($config->getEnv() === 'development') {
+ if ($this->config->getEnv() === 'development') {
 return new DevImplementation();
 } else {
 return new Implementation();
 }
 }
}

This is a bit better already. At this point i run the test suite, to see where i broke something. To my surprise, no tests broke. (This wasn’t that much of a surprise, as there are very little tests.)

So instead of moving on the the next point i added 2 tests. One to check that for development the DevImplementation was returned, and another one to make sure in other cases the Implementation was returned.

At this point i had spent less than 10 extra minutes on this code. I found it safe to take the 30 extra seconds to remove the redundant else statement. And while we are at it, lets replace the return annotation with a proper return type

- /**
- * @return ImplementationInterface
- */
- public function createImplementation()
+ public function createImplementation(): ImplementationInterface
 {
 if ($this->config->getEnv() === 'development') {
 return new DevImplementation();
- } else {
- return new Implementation();
 }
+ return new Implementation();
 }

Now, the Config class lives in another package, so adding constants to hold the environment strings would involve adding it in that package, and updating that. Although it wouldn’t be too much extra time, this was enough for now. And whether or not this was even the correct place for this was also outside of the scope for todays work.

class ImplementationFactory
{
 private $config;

 public function __construct(Config $config)
 {
 $this->config = $config;
 }

 public function createImplementation(): ImplementationInterface
 {
 if ($this->config->getEnv() === 'development') {
 return new DevImplementation();
 }
 return new Implementation();
 }
}

So in the span of maybe 15 minutes, i fixed what i originally set out to do, cleaned up the class a bit, and added unit tests. I know that there are still issues with this piece of code, and probably with the code that uses this class. So while this code isn’t perfect as of right now, it is better than how i found it. And that is what the boy scout rule is about.

Now, this was piece of code was relatively easy to refactor. I could see what changes could be made without too much work. Other times this may not be as simple. The key is to make improvements that can be made within a small time frame, and that are relatively easy to do.

One of the first improvements i would suggest to any piece of code is adding tests. Having tests to validate the behaviour of your code makes it easier to make changes. And even during that, remember the boy scout rule. 100% coverage may be too high of a goal for a specific class or method. But we can at least make the first step. And then the next time, we can improve that coverage a bit more.

Now if you consistently apply this rule, over time you can make big improvements. Initially the changes are small. But if a lot of small changes add up. And after while, you may even be able to make bigger changes, because the code is clean and understandable enough for it. And before you know it, the small change you apply, does make the code ‘perfect’.

When to add comments

Thu, 31 Oct 2019 21:16:05 +0100

A lot of methods have comments, either explaining what they do, how they do it, why they do it. And a lot of times they are wrong. More often than not, you don’t need to add comments, so lets take a look at when and why comments are useful.

When we do add comments

A valid reason to add a comment may be one of the following:

It expresses something the language can’t
It is needed for certain tools
It is a non obvious work around.

It expresses something the language can’t

If we take the following method signature as an example:

function combine(array $input): string;

We don’t know what the input array is supposed to look like. Does it expect an array of integers, strings, certain objects? This is something we can’t express in PHP itself right now. But if we write it down like so, we get a better understanding:

/**
 * @param array<int> $input
 */
function combine(array $input): string;

Now you may be more used to seeing int[] as the type hint. the array<T> is the same. But the difference is that you can also say array<int,int> as a comment, which means that you expect the keys to be integers.

Another thing the language doesn’t support yet is union types, so a parameter taking an array or a string can’t be expressed yet, but with a comment we can say @param array|string. And our IDE and tooling will understand, as well as our colleagues (hopefully).

You may also be using a library like Doctrine Annotations to add even more functionality, through annotations.

It is needed for certain tools

Now i will preface this by saying that i am not a fan of littering your code base with comments that are used to instruct tools. For example @codeCoverageIgnore or @codingStandardsIgnore shouldn’t be in your code base, but they should be part of a config. However, with annotations we can support certain tools to help us with validating our code. For example, we can add great support for verifying immutability by using Psalm and the @psalm-immutable annotation.

It is a non obvious work around.

This is the ‘real’ reason you want to use comments. You may be doing something that makes very little sense, because you are facing a bug that you can not fix.

For example, a project i worked on could be used by using only the keyboard. However, we ran into a bug on firefox where a e.preventDefault() didn’t stop a dropdown from still being changed when an arrow key was pressed. The only way to do so was to disable the element. So that is what we did. But if someone would see the code, their first thought may be that it isn’t needed. (Because it isn’t on other browsers).

So we added a comment to the method:

/**
 * We temporary disable the selector, so the pressing of arrow keys doesn't change the rating given.
 * This is needed for firefox, as the event.preventDefault() does not disable the arrow keys on selectors.
 *
 * @see https://bugzilla.mozilla.org/show_bug.cgi?id=291082
 */
function temporaryDisableSelector()

Looking back this may not be the greatest comment. Part of it just copies the function name, but it does explain why it is there, and even has a link to a bug tracker.

When we don’t add comments

TL;DR: anything else.

A lot of comments fall in the following categories:

It duplicates the code
It duplicates VCS
It is wrong

It duplicates the code

Far too often i see comments like this:

/**
 * CheckoutController class
 *
 * @package \Website\Checkout
 */
class CheckoutController

When this exact comment is on every single class in your code base, people ignore it. Now the one time you do actually have something important to say in your comment, no one is going to read it. Because everywhere else they can ignore those 5 lines, so why start reading them now.

Another one i see a lot looks something like this:

/**
 * @param int $count
 * @param string $name
 * @param Order $order
 * @param $other
 */
function doTheMagic(int $count, string $name, Order $order, $other): bool

This comment does absolutely nothing, besides duplicating the method signature. Once again, if this is part of every doc comment, people will never read it, and when there is something there people will not see it.

Sometimes the comment may appear to be relevant, but they still aren’t:

$price = null;

foreach ($order as $order) {
 if($order->hasPrice()) {
 $price = $order->getPrice();
 // Stop after we find a price
 break;
 }
}

In this case, the comment just tells you what the break tells you. You are repeating the code in a comment. The danger here is that when your code updates, the comment may be wrong. But we don’t know for sure.

For example, a few months later it may look like this, and then our comment isn’t always true, and it is also in the wrong spot.

$price = 0;

foreach ($order as $order) {
 if($order->hasPrice()) {
 $price += $order->getPrice();
 // Stop after we find a price
 $price *= $order->getDiscount();

 if(!$this->combinePrices) {
 break;
 }
 }
}

It duplicates VCS

Far too many projects are filled with @author annotations, or the default PHPStorm annotations when you create a new file, where the creation date is in the comment. Those don’t add any value, as the information can be retrieved from your version control.

It is wrong

This is the danger with any comment. Generally comments don’t start out wrong, but the code around them updates. And the comment doesn’t get updated to match. I gave an example of this in the part about comments that duplicate code.

One i saw recently looked somethign like this:

/**
 * Retrieves the information as follows:
 * 1. From the cache
 * 2. Otherwise from redis
 * 3. Otherwise it checks the config file
 * 4. From the database
 * 5. Or an empty string as a fallback
 */
function getQuickData(string $input): string
{
 return $this->sql->getFromInput($input) ?? 'Error';
}

Now looking back in the VCS, the comment wasn’t even correct when it was added, as the config file was never checked. But over time the method was refactored multiple times and the system changed, and now it just retrieves from the database.

But if we just check the comments when we peek at the method, we thing we may retrieve from cache, and we need to check for an empty string.

In conclusion

There are valid reasons for comments, but more then likely, your comments aren’t adding any value. And the more useless comments there are, the less likely the important comments are to stand out. So do consider removing comments that don’t add anything. Personally i am a fan of the Doctrine coding standard as one of its rules gets rid of useless comments for you.

Now there are more reasons to add, or not add comments to your code base. What reasons do you have for adding comments, or getting rid of them? And what are some examples of really good, or really bad comments you have seen? Do let me know in the comments.

PHP is already strictly typed

Fri, 16 Aug 2019 13:25:13 +0200

PHP is already strictly typed, in the same way that JavaScript is. Not through the language itself, but with the help of tooling.

JavaScript achieves this through tools like Typescript. (I use typescript as an example, as that is what i normally use.) Typescript adds a lot of new syntax to tha language, which allows for type checking. The transpiler then simply won’t transpile if there are type errors( depending on your configuration). Your code doesn’t do these checks during runtime, so technically you could still create errors by not using the transpiler and writing the JS code directly.

PHP achieves its strict typing through tools like psalm and phpstan. For PHP this doesn’t mean new syntax and code being transpiled however. Since for PHP transpiling the code isn’t something that is done often. There are tools for it, but its generally not that popular. However, PHP devs do love their annotations. A doc comment with @var Foo\Bar here, another one with @return string there and so on. With previously mentioned tooling we can have additional annotations. We can even have @template annotations to get generics.

Just like typescript, there is no run time validation, and because the code isn’t transpiled either, there is no way to stop someone from writing bad code. The only way to stop this is to make the success of these tools mandatory. This means running them in CI, and not allowing a merge to happen unless these tools run successfully.

We don’t need a new language/dialect like P++, we can already increase type safety by using the tooling created for that. So if you want more type safety in your code, and find errors before running the code, use these tools that are already available. If you don’t want to use these tools, then you don’t have to. And i think that is the PHP way.

The modern PHP developers toolbox

Fri, 26 Apr 2019 21:03:21 +0200

There are a lot of tools available to a developer these days. Some are well known, but there are also some diamonds in the rough. So today, we are gonna look at some of the tools a modern (PHP) developer should use. A lot of these tools aren’t specific to PHP, so even if you are using another language, this may be worth checking out.

Dependency manager

I’d like to think that most projects these days use composer these days. It has been the standard for PHP development for the past few years. Managing your dependencies is easy now!

A problem before composer what that you had to include all 3rd party code manually. And if that code has a dependency on another package you had to include that as well. Then if multiple package depended on the same package you needed to make sure you only had 1 version of that package, and the correct one. Composer does all of that for you through your composer.json in the project.

This file specifies the version ranges your project (or package) can work with, and then installs the best versions of all packages according to all the constraints that each package has. Doing so produces a composer.lock file as well, which ’locks’ you into the specific dependency versions you install. This means that wherever you install this project, as long as you have the same composer.lock, the same dependencies get installed.

Now if newer versions of the packages you use get published, you can run composer update to update to the latest versions. You can do so locally, test the changes, and then you only have to push the new composer.lock in order for the newer versions to install on production.

Code style fixer

I’d like to note that i’m talking about a fixer here, not a linter. There is a small difference, but to me the result is huge. A linter will simply tell you all the errors you have, a fixer will fix then (automatically) for you.

One thing that annoys me to no end is a tool that tells me a bracket should be on a new line. If the tool knows that the bracket is in the wrong position, and it knows the correct position, it should change it for me. There should also be the option to run it without actually fixing the issues, to be used in a CI environment for example.

The fact that PHP-CS-fixer only has ‘rules’ that it can automatically fix, makes it my favorite fixer. Every error it will tell you about is one it can fix automatically. Do you want to adhere to PSR-2? simply add the @PSR2 rule to the configuration, and the next time your the fixer, your code will be PSR-2 styled. The fact that every issue also needs to be fixable does make it more limited in what it can detect.

Another great fixer is PHP_CodeSniffer. It comes with 2 commands: phpcs and phpcbf. The first command, phpcs which finds all errors and reports them to you. The second command is phpcbf, which fixes all the errors it can, and reports the other errors to you. This means that it can be a fixer for all the issues it can solve, but still allow more complex rules, that can’t be automatically fixed. If you want a strong set of rules I’d suggest the Doctrine coding standard. These are a set of rules, used in the doctrine projects, that uses PSR-2 as its baseline, but adds a whole lot of extra rules that help you write maintainable code.

Static analyzer

The previously mentioned tools aren’t ‘modern’ per se, but they are still ever evolving. Static analysis however, is relatively new in the PHP-scene. PHPStan saw its initial release on 2016-07-16. PHAN saw its first release on 2015-12-03. And Psalm became open source on 2016-11-21.

Static analysis is in essence a type checker. But with the big benefit, of not having to run the code to find out if it works. This means that in order to use a static analysis tool, you don’t have to write tests to execute the code. Take a look at the following example:

<?php

function sayHello(): int
{
 return 'hello';
}

/**
 * @return int
 */
function sayGoodbye()
{
 return 'good bye';
}

Now the first function will crash the moment it runs. The second function however, will run normally without crashing. Except that you think you have an integer, when in reality you are dealing with a string. Running a static analyzer, both of these functions will generate an error, without executing any line of code. And because it does to without needing to execute(run) the code, it can do so for the entire code base, without needing to write any tests.

The more you use type hints, whether its in comments or in code, the more static analysis will do for you. Most of these tools work with levels, where the higher levels will be more strict about your code. For example, it will tell you that your getcwd() may return false under specific circumstances, meaning you need to check for that. Static analysis helps you with more defensive programming.

These days my goto tool is PHPStan. It has extensions for frameworks, and allows for easy configuration by using the levels it provides. I have also been using Psalm as of late which seems to have a lot more options for specifying array/iterable types. I don’t have much experience with PHAN, but its focus lies on reducing false positives, and has more tooling for checking supported php versions.

And honorable mention also goes out to Exakat, which has a lot of focus on checking against certain types of php-versions. It also checks against common mistakes and code smells.

Static analysis is easy to set up, and the minimum levels are usually not a lot of effort to fix. Starting with static analysis can give you some quick-wins, and help you write more maintainable, robust code.

Unit testing framework

Yes you need to test your code.

The biggest php unit testing framework is PHPUnit. It allows you to set up your test conditions, execute the code, and then assert if everything is working correctly.

Unit testing on is a big topic on its own which will be covered in another upcoming blog-post

Mutation testing framework

I covered mutation testing, or more specifically infection in a previous blog-post, which I would recommend you to read. It is rather new, just like static analysis. Humbug saw its first pre release on 2015-05-25. And infection had its 0.1.0 release on 2017-07-01. Humbug has since been deprecated in favor of infection.

Mutation testing is an extension of unit testing. It is an extra layer on top of your mutation testing that makes sure your tests are catching the things they need to. Mutation testing is ’testing your tests’.

Editorconfig

Okay, this one isn’t really a tool of modern PHP developers, but I like editorconfigs, and its my blog.

Editorconfigs are a configuration file that you add to (the root of) your project, which tell your editor what kind of indenting to use where etc. It also tells your IDE whether or not to insert a final new line, clean up trailing white space etc.

Its configured in an ini like matter, and most IDE’s honor it out of the box, but some need a plugin/extension in order to use it. I’d reccomend checking out the editor config website to see all the options and how to use it.

Closing thoughts

This isn’t a full list of the tools you should be using, and some tools may be useless in your project or setting. I do consider this to be a baseline of tools I would use for any project. I didn’t include IDE’s or version control like git, but i do consider those vital.

Do you have tools you consider vital for development that aren’t mentioned here? Or do you disagree with the tools mentioned here? Let me know by leaving a comment, or by sending me a DM on twitter.

What Is Big O

Tue, 12 Feb 2019 00:00:00 +0000

Big O is used to notate the ’time and space complexity’ of algorithms. It provides us with key insights into how fast an algorithm is, without getting bogged down with details.

Calculating the work of algorithm

Lets start with an example, consider bubblesort, which looks something like this in python:

def bubble_sort(arr):
 n = len(arr)
 # Traverse through all array elements
 for i in range(n):
 # Last i elements are already in place
 for j in range(0, n-i-1):
 # traverse the array from 0 to n-i-1
 # Swap if the element found is greater
 # than the next element
 if arr[j] > arr[j+1] :
 arr[j], arr[j+1] = arr[j+1], arr[j]
 return arr

It loops over all the elements in the first loop, and then loops over all elements again (save the last i items), swapping any element if the next one to it is higher, to make sure the highest entry ends up in the most right position. Which means that for every item in the array, it loops over the entire array again, save a few items.

Now how can we express, in numbers now much ‘work’ this algorithm does, relative to the length of the array, $n$?

It loops over the entire array once in the first part, and then once again over every item, once more in the inner loop. For every item in that inner loop, it does a check, and then swaps 2 items if it needs to. So we could say that it does, at most $3n^2$ work. Now, it does calculate the length at the start of the function, and it does increase j every time, so thats a bit of work too. Meaning that the work could be expressed as $4 n^2 +1$. And to make this a bit more easy, i’m forgetting that after every run of the outer loop, the inner loop has to run one less time. So lets keep it at $5n^2 +1$ work. We can also call the amount of work the ’time complexity’ of an algorithm.

Now, what if we slightly change this code, maybe optimize it a bit, then the amount of work would change. But the core of the bubble sort still remains the same, we still loop over the entire array, for each item in the array. Or what if we implement it in a different language? Having to know all details of what the compiler translates it into in bytecode would be a hassle. So lets simplify how we calculate the ‘work’ of the algorithm.

In comes Big O

Big O provides us with a notation, which gives great insight in the efficiency of an algorithm, and how well it scales, without dealing with the details. Lets start with the calculated work of bubble sort in our previous example. ($5n^2 +1$). If we have an array of $n=100000$, then does the $+1$ really matter? At that point it has a negligible impact. And big number is what Big O is about. Because if we have an array with 7 elements, then it does not matter how efficient an algorithm sorts, because it will be done near instant regardless.

So we are left with $5n^2$. But, as mentioned previously what if we work in another language, or implement it slightly different, then do we need to run the calculation again. And we want an indicator of how efficient our algorithm is, without it being dependent on language. So in Big O, we can drop leading constants. Meaning the Big O of our bubble sort algorithm is $O(n^2)$.

Why Big O

While Big O does remove a lot of details, it does so for a good reason. Big O tells us how well an algorithm scales. An algorithm with Big O of $O(n)$ scales a lot better than one of $O(n^2)$. Doubling the size of the input in the first case, will double the time it takes, while in the second case, the time quadruples.

When not to care about Big O

There are a lot of good reasons to care about the Big O of the algorithms you use. However if you need to sort an array of 50 items, then no matter what algorithm you use, it will be fast. Big O only starts to matter on bigger input sizes. If you need to deal with a few thousand items, then there can be a noticeable difference between $O(n \cdot log(n))$ and $O(n^2)$.

Generally speaking, the lower the Big O, the better, but don’t let that be the only fact on which to base your decision. As stated before, sometimes two algorithms with the same Big O can have a big difference in the run time. So be sure to do some profiling and see which works best for your use case.

Closing thoughts

I left out a few steps on Big O, to keep this a bit simpler. For example, lower order bounds are also ignored in the Big O, and i didn’t go into detail on the mathematical definition of Big O, or how to prove its calculation. But i hope it did give you some insights in what Big O is, and how you can use it to decide on what algorithm to use.

Running Homestead from a separate directory

Wed, 06 Feb 2019 00:00:00 +0000

When creating a PHP website, there is a good chance that you wish to use a virtual machine. This has many benefits, like managing multiple versions of php, or extensions etc.

A lot of projects use in combination with . But nobody likes to set up a Vagrantfile every single time. Laravel created , which deals with all the vagrant configuration for you. You can install it in your project with composer install laravel/homestead, and you got it.

But here comes the problem. If you have a linux VM, and have a windows ‘host’ system, then you will have to do the entire composer install twice. Once on the ‘host’ system, and once again inside the box. Plus if you use older versions of symfony, it could conflict with homestead.

Instead, we can install it in another directory.

So inside the project, create a new folder (We’ll use homestead), and cd into it cd homestead. There run a composer init, and specify that you want to use laravel/homestead as a dependency when asked. The rest of the file doesn’t really matter. Now run a composer install, inside this directory.

After that you can go back to the root folder cd .., and then run the following composer command: composer install --working-dir homestead. After that you can run ./homestead/vendor/bin/homestead make. This will add a few files to the root of your project.

One thing we will need to update now is the Vagrantfile.

# -*- mode: ruby -*-
# vi: set ft=ruby :

require 'json'
require 'yaml'

VAGRANTFILE_API_VERSION ||= "2"
confDir = $confDir ||= File.expand_path("vendor/laravel/homestead", File.dirname(__FILE__))

homesteadYamlPath = File.expand_path("Homestead.yaml", File.dirname(__FILE__))
homesteadJsonPath = File.expand_path("Homestead.json", File.dirname(__FILE__))
afterScriptPath = "after.sh"
customizationScriptPath = "user-customizations.sh"
aliasesPath = "aliases"

require File.expand_path(confDir + '/scripts/homestead.rb')

# rest of the file ...

Lets update one key line:

- confDir = $confDir ||= File.expand_path("vendor/laravel/homestead", File.dirname(__FILE__))
+ confDir = $confDir ||= File.expand_path("homestead/vendor/laravel/homestead", File.dirname(__FILE__))

If you did everything correctly, your project structure should look something like this:

\homestead
 composer.json
 composer.lock
 vendor/
after.sh
aliases
composer.json
composer.lock
Homestead.yaml
Vagrantfile
vendor/

If you updated this from an old repository, be sure to remove the old laravel/homestead dependency from the composer.json. And update the documentation to run composer install --working-dir homestead to get the homestead dependency, instead of the normal composer install.

Take a look at to see it in action, and for more in depth homestead configuration, see

Versions used:

Composer: 1.8.3

Homestead: 8.0.2

Math Is Fun

Wed, 06 Feb 2019 00:57:15 +0100

$\sqrt{x} = \frac{x}{\sqrt{x}}$

Seeing the above equation may look completely logical, or not. When i saw it a few days ago, i thought it was wrong. When i understood that it was correct, i thought it was the most beautiful thing ever. I’m not 100% sure why this intrigued me so much, but it just looks great.

There are a few ways of simplifying this. We could multiply both sides with $\sqrt{x}$, like so:

$\sqrt{x} \cdot \sqrt{x} = \frac{x}{\sqrt{x}} \cdot \sqrt{x}$

$ x = \frac{x}{\sqrt{x}} \cdot \sqrt{x}$

$ x = x $

Another way would be to do cross-multiplication

$\frac{\sqrt{x}}{1} = \frac{x}{\sqrt{x}}$

$\sqrt{x} \cdot \sqrt{x} = {x} \cdot 1 $

$ x = x $

And there are probably a ton of other ways to effectively ‘solve’ this. But if there is one thing that i learned from this simple calculation, is that i actually like doing this. I’m having fun with math again!

Semantic versioning

Fri, 09 Nov 2018 00:00:00 +0000

Chances are, if you are writing software, you have some dependencies on other peoples code. There is no reason to reinvent the wheel, so you use the code someone else wrote. One of your options is to copy paste it, but then you won’t get any updates if they release a new version.

This is where dependency managers come into play. With a set of instructions, they retrieve the needed dependencies for you, and allow you to lock into specific versions, and update when you want. Before we head into dependency managers, lets first talk about semantic versioning (semver).

Semver

There is a website dedicated to explaining what semver entails. For now the most important parts are the following:

Version numbers look as follows: MAJOR.MINOR.PATCH
Any ‘breaking’ change is a Major version up
Any (set of) new feature(s) is a Minor version up
Bug fixes and fixes with no impact (refactoring/fixing typos) are a Patch version up

This is only for project that have a public API, and is only relevant to the public API. Which means that if you expose a class, for other developers to use, then you can change the private parts of it without a major version bump. But removing a public method, that the users may call is a breaking change, which requires a major version up.

Dependency management

Lets take symfony as an example. Let say we have a project, and need the features from symfony/console 3.4. In our composer.json we set the required version like so: "symfony/console": "^3.4". What we tell composer is the following: We need version 3.4, or higher. But not past a major. This means that it will attempt to install the highest 3.* version possible, but at least 3.4.0. So if version 4.0.0 will be released, it will not be installed, because that can contain breaking changes.

Before 1.0.0

Semver works a bit different before a version 1.0.0 is released. Before this version, every MINOR version up is allowed to have breaking changes, as they are considered pre-releases. This means that if we take the same example as above but like so: "symfony/console": "^0.4", then we can only install versions of 0.4.*. Meaning that version 0.5.0 will not be installed, as it can contain breaking changes.

This is something that is easy to forget, and may cause you to wonder why you aren’t updating to the newest version of a package. So for pre version 1.0.0 packages, be sure to check their updates. There may still be breaking changes between releases, so be sure to keep up to date with their changes.

Add an editorconfig to your project

Fri, 17 Aug 2018 00:00:00 +0000

One of the hardest things to spot during a pr are whitespace issues. Did someone use tabs instead of spaces, trailing whitespace etc. So, why not make it easy on yourself and help any contributors by adding an .editorconfig file that automatcially fixes those things for you.

An editorconfig is an ini like file that helps your IDE or text editor to keep certain standards. Even this blog, which is written in mostly markdown has the following editorconfig:

root = true

[*]
charset=utf-8
end_of_line=lf
trim_trailing_whitespace=true
insert_final_newline=true
indent_style=space
indent_size=4

root=true means that it is the top level editorconfig. This is usefull if you have certian directories that have a different style for whatever reason. The charset is set to utf-8, we use ’normal’ line endings, and trim all trailing whitespace. All files have an empty new line at the end, and are indented with 4 spaces.

If i have a Makefile, which needs to be indented with tabs i can add the following:

[Makefile]
indent_style=tab

And now my makefile is correctly indented with tabs.

If the project contains js and scss files that i want indented with 2 spaces i can add the following:

[*.{js,scss}]
indent_size=2

The editor config website has an explanation on what options there are, and how to specify what files to affect etc. Some editors check for a .editorconfig file by default, while other need a plugin to understand and use them. You can find the full list on their website as well. (But PHPstorm needs a plugin, so please do install it.)

What is a default object?

Fri, 10 Aug 2018 00:00:00 +0000

What is a clean solution to display a ‘default’ message to the user, when something they try to access isn’t there (anymore)? We could let the our repository throw an exception, catch it somewhere, and then let our controller handle it. Maybe we could return null, and pass that all the way up and add a fall back message for the content in the view somewhere? Why not try a better solution and work with a default object.

A default object (or null object) should come from business requirements. Lets say you are writing a website for bloggers. If the user goes to see a blog post that doesn’t exist(anymore), it should display the error message: Sorry, no blog post was found on this url. There are multiple ways to do this, so lets first look at doing this the ‘old fashioned’ way and throw an exception somewhere:

//Blog.php

class Blog
{
 public static function fromDB(Entity $blogEntity)
 {
 //logic
 return new Blog($title, $content));
 }

 public function getTitle(): string
 {
 return $this->title;
 }

 public function getContent(): string
 {
 return $this->content;
 }
}

//BlogService.php

class BlogService
{
 /**
 * @param int $id
 *
 * @throws BlogNotFoundException
 *
 * @return Blog
 */
 public function getBlog(int $id): Blog
 {
 $blogEntity = $this->db->getById($id);

 if($blogEntity === null) {
 throw BlogNotFoundException::notFoundById($id);
 }
 return Blog::fromDB($blogEntity);
 }
}

//BlogController.php

class BlogController
{
 public function showBlog(int $blogId)
 {
 $blg = $this-blogservice->getBlog($blogId);

 try {
 return $this-render('blog.show.html', [
 'title' => $blog->getTitle(),
 'content' => $blog->getContent(),
 ]
 )
 $blog = $this->blogService->getBlog($blogId)
 } catch (BlogNotFoundException $e) {
 return $this-render('blog.show.html', [
 'title' => 'Sorry, no blog post was found on this url'
 'content' => '',
 ]
 }
 }
}

Chances are that you’ve seen code like this before. Or maybe the service would return null, and then the logic of displaying the title or the default text was placed somewhere in the view. This means that multiple places must ‘care’ about whether or not the blog post exists. It also causes the requirements to be hidden somewhere within your infrastructure. It may make it harder in case the business decides they want a different fall back message.

Now lets refactor the code to use a default object, and see what that looks like.

<?php
//Blog.php

class Blog
{
 public static function fromDB($blogEntity)
 {
 //logic
 return new Blog($title, $content);
 }

 public static function fromNotFound()
 {
 return new Blog('Sorry, no blog post was found on this url');
 }

 public function getTitle(): string
 {
 return $this->title;
 }

 public function getContent(): string
 {
 return $this->content;
 }
}

//BlogService.php

class BlogService
{
 public function getBlog(int $id): Blog
 {
 $blogEntity = $this->db->getById($id);

 return $blogEntity ? Blog::fromDB($blogEntity) : Blog::fromNotFound($id);
 }
}

//BlogController.php

class BlogController
{
 public function showBlog(int $blogId)
 {
 return $this-render('blog.show.html', [
 'title' => $blog->getTitle(),
 'content' => $blog->getContent(),
 ]
 }
}

Now to me, this looks a lot cleaner. The Controller no longer knows if the blog was found or not, and when you read the code it is still clear what is happening. We use a named constructor here, which makes it clear what is going on. We could also opt to use a separate class entirely. One of the benefits of doing so is that you could place in your Domain namespace since this deals with domain logic (if you are separating into domain and infrastructure). I personally prefer to make all classes final or abstract, so i use a named constructor.

An implementation with a separate class may look something like this:

<?php
//NotFoundBlog.php

class NotFoundBlog extends Blog
{
 public function getTitle(): string
 {
 return 'Sorry, no blog post was found on this url';
 }

 public function getContent(): string
 {
 return '';
 }
}

//BlogService.php

class BlogService
{
 public function getBlog(int $id): Blog
 {
 $blogEntity = $this->db->getById($id);

 return $blogEntity ? Blog::fromDB($blogEntity) : new NotFoundBlog();
 }
}

If you enjoyed this post, have comments, feedback, or anything else, feel free to comment, or hit me up on twitter

Noop polyfills

Sun, 05 Aug 2018 00:00:00 +0000

A while ago a reddit post showed up, where someone installed version 9.99.99 of paragonie/random_compat. Seeing a package update from 2.* to 9.99.99 may be a bit confusing, but given how autolaoding, polyfills and composer work in php, this is actually quite a clever way of dealing with things. Lets take a look at version constraints, autoloading and composer to see why.

Applying a polyfill

The purpose of the polyfill in question is to provide random_* functions to php versions below 7.0. As these functions were only introduced in php 7.0. You can’t declare these functions yourself in versions over 7.0, as it has already been declared by php itself. Resulting in an error like: Fatal error: Cannot redeclare random_bytes() in /foo.php on line 6.

The file that gets loaded therefore first checks the php version. In other cases, like polyfilling an extension, it would check the existence of that function, or if the extension is loaded.

Autoloading

A composer.json with a section something like this may be familiar.

{
 "autoload": {
 "psr-4": {
 "Acme\\Foo\\": "src/"
 },
 "files": [
 "src/Helpers.php",
 "src/global-functions.php"
 ]
 }
}

We use the psr-4 standard for autoloading, and we load a few files as well that contain global functions/helpers. I won’t go into detail about psr 4 as that isn’t relevant for this blog post. But lets look at how the files are autoloaded by composer.

If we dive into our vendor/composer/autoload_real.php we will find something like the following. The class names may differ as they are auto-generated, but it should generally look like this.

The $includeFiles will be all the files that are in the autoload.files part of composer.json (as well as the composer.json files from packages you depend on). We can see that if the file hasn’t been required already, its required. This is done as soon as you include/require vendor/autoload.php, which is generally done at the start of every process.

<?php
class ComposerAutoloaderInit0e3a5124969d920e9ee3b981a725692a
{
 //...

 public static function getLoader()
 {
 //...

 if ($useStaticLoader) {
 $includeFiles = Composer\Autoload\ComposerStaticInit0e3a5124969d920e9ee3b981a725692a::$files;
 } else {
 $includeFiles = require __DIR__ . '/autoload_files.php';
 }
 foreach ($includeFiles as $fileIdentifier => $file) {
 composerRequire0e3a5124969d920e9ee3b981a725692a($fileIdentifier, $file);
 }

 return $loader;
 }
}

function composerRequire0e3a5124969d920e9ee3b981a725692a($fileIdentifier, $file)
{
 if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
 require $file;

 $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
 }
}

This means that if you have paragonie/random_compat as a dependency, even if its an indirect dependency, you will require its bootstrap file every time. Which means that ever process starts with a check to see if you have php 7.0 or not.

Noop versions

This is where the noop version comes in. The version that does ’nothing’. It doesn’t populate the autoload, thus the files are never required. It has a really high version, so that you can require ~1.0||~2.0||~9.99.99. Composer will always attempt to install the highest possible version of a package. So if you have php 7.0 or higher, it will install version 9.99.99, and if you don’t it will attempt to install a lower version. This is currently the best way a package can declare to ’not’ be installed if certain criteria are met.

So if you have a package that uses paragonie/random_compat, and you are installable on php 7.0+, update the requirements to 9.99.99. If you are not compatible with php below 7.0, and you get paragonie/random_compat from another dependency which doesn’t allow 9.99.99 to install, consider adding the following to your composer. This will completely remove the package from your dependency tree. You shouldn’t do this if you are compatible with lower php versions.

{
 "replace": {
 "paragonie/random_compat": "*"
 }
}

If you are directly using a symfony polyfill, you may soon also be able to require a version 9.* that does the same thing, as it may soon become available.

If you wish to know more about polyfills and extensions, check out my other blog post about polyfills and exentions.

Deploying to Github pages with Travis

Tue, 31 Jul 2018 00:00:00 +0000

If you want to write a blog, or have another kind of static website, you need to deploy it somewhere. So why not host it on Github pages? It’s free and has https by default.

To deploy a website that gets generated, for example by jekyll, we need to make a few steps before we can deploy it to Github pages.

We need the following things:

Two Github repositories
Set up Travis

Two Github repositories

Because we want the code that ‘creates’ the website, and the website itself on Github, we need two repositories. The first one can be whatever you wish to call it. In this example the repository will simply be called Blog. The second repository, which holds the website needs a special name. This repository will be called <Your github user name>.github.io. So in my case it is called BackEndTea.github.io.

Once you have added some content to the Blog repository, and want to show it to the world, follow the next steps.

Set up Travis

Thankfully, creating a Travis account is as easy as going to their website, and clicking the Sign in with Github button. There you will find a list of all (public) repositories you have access to. Activate the one for Blog like so:

Now lets add a file called .travis.yml to the root of our Blog project, which will look like this:

language: ruby
cache: bundler

branches:
 only:
 - master

install:
 - bundler install

script:
 - bundler exec jekyll build

deploy:
 provider: pages
 skip_cleanup: true
 local_dir: _site
 github_token: $GITHUB_TOKEN # Set in travis-ci.org dashboard
 keep-history: true
 on:
 branch: master
 repo: <Your-github-name>/<Your-github-name>.github.io
 target_branch: master

Replace the language/install/script parts with what is relevant for your tool. Since this example uses Jekyll we need ruby, install the project with bundler, and run bundler exec jekyll build to build the project, which puts it in the _site folder by default.

The deploy part is a bit more interesting. The provider key is pages, as we are deploying to Github pages. We set skip_cleanup to true, as this would otherwise ‘clean up’ the freshly build website. local_dir is the folder to actually deploy. If we set keep-history to false, every push will be force pushed, and the repository will have no git history. The repo is the Github repository to deploy to, which we created in the first step. And we deploy to master.

Now for the github_token. We go back to Github and go to our settings like so:

There you can go to Developer settings, and then to Personal access tokens. Click the Generate new token button, and give it the following permissions:

This should be the only permissions it needs.

Once you the token it will show you the token one time only, so don’t close the tab yet. Move back to Travis and go to the settings of your repository. There you will see a section called Environment Variables. On the left is the key to access the value, and the right is the value itself.

The key here will be GITHUB_TOKEN, which is the same as we put in our .travis.yml, without the leading $. The value is the token we just generated. Once you pasted it to Travis you will lose the ability to see the token. Thankfully, we no longer need the token ourselves, as from now on Travis will deal with the deployments for us.

With our current settings, every time we push to master, our website will be re-build and deployed to Github pages automatically. As a note, don’t re-use your github tokens, and only give it the permissions it needs, and if you no longer use it, delete it. If the token gets compromised it could be used to make changes to your repositories and do a lot of damage.

Infection 0.9 is out!

Wed, 25 Jul 2018 00:00:00 +0000

I wrote about infection a while ago, and not too long after that, 0.9.0 was released. Lets look at the new features, and how we can use them in our projects!

Profiles

Previously, if you wanted to run all mutators, except for one, it would involve passing all mutators in the commandline, except for the one you didn’t want, which was tedious. With the addition of profiles this became a whole lot easier.

Here we have a simple configuration file, where we use all mutators, except for the @function_signature ones (PublicVisiblity & ProtectedVisibility).

{
 "timeout": 10,
 "source": {
 "directories": [
 "src"
 ]
 },
 "logs": {
 "text": "infection-log.txt"
 },
 "mutators": {
 "@default": true,
 "@function_signature": false
 }
}

Instead of a profile (which always starts with an @ and is snake_case), you can also disable a single mutator directly. e.g. "PublicVisiblity": false.

Disabling mutators for specific places.

Besides passing true or false to a mutator in the config, we now have the ability to add specific settings to it.

Lets say we want to disable the Plus mutator for the method named doPlus in the Foo\Bar\Baz class, we can do it like so:

{
 "mutators": {
 "@default": true,
 "Plus": {
 "ignore": [
 "Foo\\Bar\\Baz::doPlus"
 ]
 }
 }
}

But what if we want to disable the Minus mutator, for all classes in the Foo\Bar namespace, and the @function_signature profile for all methods named noTrue in the Foo namespace?

It can easily be done by adding the following parts to your configuration file.

{
 "mutators": {
 "@default": true,
 "Minus": {
 "ignore": [
 "Foo\\Bar\\**"
 ]
 },
 "@function_signature": {
 "ignore": [
 "Foo\\**::noTrue"
 ]
 }
 }
}

Important: If you want to (partially) disable certain mutators or profiles, but still run everything else, be sure to add "@default": true" to your config, which enables all the other mutators.

Mutation Badge

Thanks to the people who made Stryker, you can now add a new badge to show off your mutation score. You can find the instructions right here.

Smarter mutators

This mutation would always result in the following error, making it a useless mutation.

Fatal error: Access level to B::foo() must be public (as in class A) in /foo/test.php on line 14

abstract class A
{
 abstract public function foo();
}

class B extends A
{
- public function foo()
+ protected function foo()
 {
 }

}

Infection now uses reflection to determine if a method has a parent class or an interface that declares the same method. And it does not mutate the visibility if the method is inherited. If you use a lot of abstract/parent classes and interfaces this could drastically reduce the amount of mutations you get, and thus speed up the process by a lot.

Phar Scoping

Now you may be worried, that autoloading your code, when using a phar could result in a clash while autoloading. e.g. your project has a dependency on Symfony/Console, and the phar does as well, so which one does infection load? This has been fixed by using PHP-Scoper, which prefixes all namespaces within the phar with a random string, and makes it so that the code, and autoloading, within the phar can not clash with the code in your project.

For example, your project would use (and thus autoload) the following class Symfony\Component\Console\Application. Now infection also uses this class, but within the phar its called something like: ScopedRandomgString\Symfony\Component\Console\Application. This means that infection has a separate version of the class that it uses.

PHP-Scoper a really cool project, so if you are interested in it, be sure to check it out.

New mutators

Not only did mutators get smarter, we also got a whole lot of new mutators:

Finally_:

Removes the finaly block that is part of a try catch block, e.g.

try {
 dangerous_function()
}
catch(Exception $e) {
 //handle exception
}
- finally {
- // Do this no matter what
- }

This should help you test your error handling.

PregQuote:

Removes preg_quote function calls, e.g.

- $a = preg_quote('value');
+ $a = 'value';

This should help you test the usage of preg_quote calls, and maybe even show you don’t need/use them.

ArrayItem& Yield_:

Changes the => into > for arrays and yield respectively,

[
- $a->var => $b->var
+ $a->var > $b->var
]

This should help to test the usages of your arrays and yield statements

Assignment:

Changes assignments like +=. -+, .= etc into plain =.

$a = 'title;
- $a .= ' suffix';
+ $a = ' suffix';

This should help you test that assignments are done properly.

For_:

The for counterpart to the ForEach_ mutator that stops for loops from happening:

- for($i =0; $i < count($array); $i++)
+ for($i =0; false; $i++)

This should help testing that your for loops are actually happening.

Updating

If you use composer to add infection as a dev dependency, be sure to change the version restriction in your composer.json to ^0.9.0, and hit composer update. Because in pre 1.0.0 releases composer won’t bump minor versions, as they could have breaking changes.

If you are downloading the par, simply change the current version in the url to 0.9.0 and you should be good.

In conclusion

Infection 0.9 came with a lot of improvements, especially for (partially) disabling mutators or groups of mutators. The mutators also got a bit smarter, and hopefully you should see your mutation testing being done a lot quicker.

If you liked this blog post, and/or have questions/tips/whatever, feel free to leave a comment here, or hit me up on twitter

PHP will lose its LTS

Tue, 24 Jul 2018 00:00:00 +0000

In about 5 months PHP 5.6 loses its security support. Which means that in 5 months, PHP loses its current Long Term Support(LTS) version. If you don’t count security only support as LTS, then PHP hasn’t had an LTS for the past 1.5 years. If your website can’t handle PHP 7 or higher, you may start finding it harder to get support for that, with hosting sites like acquia removing php 5.6 support We will look at the advantages, and disadvantages of LTS, and what it means for the PHP ecosystem.

What is LTS

In essence, LTS is exactly what it says it is. A specific version gains longer that usual support. Take Symfony for example. Every release has 8 months of active, bug fixing, support. After that it has 6 months of security only support, and after that the version has no official support. The LTS versions (3.4, and 4.4 etc. in the future) have 3 years of bug support, and then 1 more year of security only support. Generally, there will also be a clear upgrade path between the latest LTS and the current one.

Why LTS

Now you may wonder, why would i support a specific version for a long amount of time, when there are newer versions with more features. The reason for that is simple. If a company uses your framework/package/programming language a lot, every backwards incompatible change could mean a lot of rework to their applications. But not updating means bugs wont be fixed and security vulnerabilities wont be patched. LTS solves both problems by keeping everything the same, and only fixing bugs and/or security issues.

Of course LTS also has downsides. For the maintainers it means that every bug fix or security patch has to be applied to the LTS version first, and then added on top of the current version. And then there is still the fixing of any conflicts that could be caused by the bug and the difference between LTS and the current version.

PHP LTS

Now PHP’s LTS is a bit different. Every minor version already has 2 years of bug fix support, and then 1 year of security only support. Which means you could already consider them LTS. Another difference is that PHP does not follow semantic versioning. Which in essence means that almost every minor update(e.g. 7.1 ->7.2) has one or more backwards incompatible changes. It may also require one to update dependencies that are incompatible with those changes, which could end up being a lot of work, as it may require updating major versions of those dependencies.

If you asked someone what version your new PHP package should support, a lot of people would have told you to support 5.6. With its LTS ending, soon, 7.0 seems to be all the new go to lowest version. The funny thing is that 7.0 will lose its security support in 4 months, which is earlier than 5.6. So some opt to make the lowest version 7.1 or 7.2.

Until PHP gains a new LTS, any new package should support the last, or last 2, versions of PHP. Unless you are filling a very specific niche, or need to stay in line with other packages, supporting 5.6 shouldn’t really be done any more. And please don’t create new packages for PHP 5.3. Now if you are already maintaining something, upgrading the minimum PHP versions is a backwards incompatible change, and means you have to bump the major version if you follow semantic versioning. If a new PHP LTS gets released we should attempt to support that version as a minimum until it gets deprecated.

Automated code reviews

Thu, 28 Jun 2018 00:00:00 +0000

Sometimes a code base has specific code style rules, that aren’t easy to detect with either with a style fixer or a static analyzer. So instead of having to tell every new contributor your rules during a PR, or lay them all out in your CONTRIBUTING.md, why not automatically check them with PHPUnit. We will look at a couple of different ‘rules’ your code base could have, and how to set up the automated tests.

The goal of these tests is not to replace code reviews. Instead it allows your code review to focus on the actual code, rather than having to read through a PR to check if classes are final, or if they don’t expose public properties. It allows you to focus on those important things, like what the code is doing, and how it is doing it. It won’t replace static analyzers either, as the goal of these tests is to automate those little, usually opinionated things, that are hard to put into a single ‘rule’ for static analysis, but instead differ greatly from project to project.

We will go through writing tests for the following cases:

Source classes must have no public properties
Source classes must be final
Source classes must have a corresponding unit test

Hopefully this should get your familiar with how to write these tests.

But before we do that, we have to gather a list of all our classes for which we can use

Getting the classes

To get our source classes, we use the symfony/finder component, and we write two methods, assuming our project uses psr-4.

<?php
//ProjectCodeTest.php

public function providesSourceClasses(): array
{
 return array_map(
 static function ($item) {
 return [$item];
 },
 $this->getSrcClasses()
 );
}

private function getSrcClasses(): array
{
 static $classes;

 if (null !== $classes) {
 return $classes;
 }

 $finder = Finder::create()
 ->files()
 ->name('*.php')
 ->in(__DIR__ . '/../../src')
 ->notName('helpers.php')
 ;

 $classes = array_map(
 static function (SplFileInfo $file) {
 return sprintf(
 '%s\\%s%s%s',
 'VendorNamespace',
 strtr($file->getRelativePath(), DIRECTORY_SEPARATOR, '\\'),
 $file->getRelativePath() ? '\\' : '',
 $file->getBasename('.' . $file->getExtension())
 );
 },
 iterator_to_array($finder, false)
 );

 sort($classes);

 return $classes;
}

The providesSourceClasses is our dataProvider, that we use on all our tests. Lets take a look at the getSrcClasses method and see what it does.

<?php
static $classes;

if (null !== $classes) {
 return $classes;
}
//...

The variable we will assign the list of classes to is $classes. By using a static variable, we can cache it within the method, so the next time we don’t have to iterate over the file system again, and simply return our cached list.

<?php
//...
$finder = Finder::create()
 ->files()
 ->name('*.php')
 ->in(__DIR__ . '/../../src')
 ->notName('helpers.php')
;
//...

We create a finder object that gets us all files ending in .php in the src directory (your indentation level may be different), except for helpers.php. In this scenario we exclude the helpers.php file as it is a file with functions, and not a class.

<?php
//...
$classes = array_map(
 static function (SplFileInfo $file) {
 return sprintf(
 '%s\\%s%s%s',
 'VendorNamespace',
 strtr($file->getRelativePath(), DIRECTORY_SEPARATOR, '\\'),
 $file->getRelativePath() ? '\\' : '',
 $file->getBasename('.' . $file->getExtension())
 );
 },
 iterator_to_array($finder, false)
);

sort($classes);

return $classes;

We use array_map to loop over all the files found by our finder, which are SplFileInfo objects, and replace change those into a Fully Qualified Name(FQN) strings of our classes. The VendorNamespace is the namespace you use before everything else. If the autoload in composer.json looks like this, you would replace it with Acme\\Foo.

"autoload": {
 "psr-4": {
 "Acme\\Foo": "src/"
 }
}

We then append the relative path, starting in the src folder, and replace our directory separators with a \, to turn it into its namespace. If the relative path is empty, its directly in the src folder, we do not add another \ since its already after the vendor namespace, otherwise, we add a new \, and then finally we append the name of the file, without its suffix. Which should turn into our class name.

The iterator_to_array is used as array_map doesn’t accept generators. We then sort it and return it to our provider.

Our provider wraps it into an array to make sure it can be used by our PHPUnit tests.

Now, the final step before we can finally write those tests to check our code style, is to make sure this provider is valid. We may have another file that doesn’t contain a class that gets picked up by our finder, or someone may add one later and run into strange errors.

<?php
/**
 * @dataProvider providesSourceClasses
 *
 * @param string $className
 */
public function test_src_class_provider_is_valid(string $className)
{
 $this->assertTrue(
 class_exists($className) || interface_exists($className) || trait_exists($className),
 sprintf(
 'The "%s" class was picked up by the source files finder, but it is not a class, interface or trait. ' .
 'Please check for typos in the class name. Or exclude the file if in the ProjectCodeTest if it is not a class.',
 $className
 )
 );
}

This simply checks if the provided class name is either a class, interface or trait. And then gives a descriptive failure message to the user.

Error Messages

As seen in the example above, half of the method is the error message. Since we want these tests to help our uses adhere to our standards, it should be clear why the test is failing. Simply seeing Failed asserting that false is true. doesn’t help our users at all.

And for what its worth, descriptive test failure messages are always a good idea, as it helps you figure out why a test is suddenly failing.

The rules

Source classes must have no public properties

<?php

/**
 * @dataProvider providesSourceClasses
 *
 * @param string $className
 */
public function test_src_classes_do_not_expose_public_properties(string $className)
{
 $rc = new \ReflectionClass($className);

 $properties = $rc->getProperties(\ReflectionProperty::IS_PUBLIC);

 $properties = array_filter($properties, function (\ReflectionProperty $property) use ($className) {
 return $property->class === $className;
 });

 $this->assertCount(
 0,
 $properties,
 sprintf(
 'Class "%s" should not declare public properties, ' .
 "if it has properties that need to be accessed, consider getters and/or setters instead. \nViolations:\n%s",
 $className,
 implode("\n", array_map(static function ($item) {
 return " * ${item}";
 }, $properties))
 )
 );
}

Lets go through the code again, and see what it does.

<?php
$rc = new \ReflectionClass($className);

$properties = $rc->getProperties(\ReflectionProperty::IS_PUBLIC);

$properties = array_filter($properties, function (\ReflectionProperty $property) use ($className) {
 return $property->class === $className;
});

We create a ReflectionClass of our class, and get all public properties.

We then filter this down to only the properties of the current class, by removing any properties that belong to parent classes. This makes sure that we don’t get a lot of errors if a parent class has a public property, or if we extend from a class outside our code base that has public properties.

Of course there might be edge cases, where you want to have public properties. For example, if you create your own stream wrapper, you need a public property named context. So if you have a stream wrapper named StreamWrapperClass, you can add the following snippet before the assertion.

<?php

if ($className === StreamWrapperClass::class) {
 // The StreamWrapperClass needs 1 public property: $context
 // @see https://secure.php.net/manual/en/class.streamwrapper.php
 $this->assertCount(
 1,
 $properties,
 sprintf(
 'The "%s" class must have exactly 1 public property as it is a streamwrapper. ' .
 'If this has changed due to recent php developments, consider updating this test.',
 $className
 )
 );
 $this->assertSame(
 'context',
 $properties[0]->getName(),
 sprintf(
 'The "%s" class must have exactly 1 public property named context. ' .
 'If this has changed due to recent php developments, consider updating this test.',
 $className
 )
 );

 return;
}

Once again, we use a descriptive message to explain to our user what is going on. We use a return statement, as we do not want to run the rest of the method, and its generally considered cleaner than an else statement.

Source classes must be final

Lets assume we want to make all our classes final (except for a few, but we’ll get back to that). This is useful when you are writing a library and don’t want users to extend your classes.

Reflection classes have a method named isFinal, but we may have abstract classes, traits and interface, which can not be final, so that won’t be enough. To get around this we write a provider that gives us only the ‘concrete’ classes.

<?php

public function provideConcreteSourceClasses(): array
{
 return array_map(
 static function ($item) {
 return [$item];
 },
 $this->getConcreteSrcClasses()
 );
}

private function getConcreteSrcClasses(): array
{
 return array_filter($this->getSrcClasses(),
 function ($class) {
 $rc = new \ReflectionClass($class);

 return !$rc->isInterface() && !$rc->isAbstract() && !$rc->isTrait();
 }
 );
}

We filter out all classes that are interfaces, abstract or traits, leaving us with just the concrete classes. And create the following test, to make sure new classes are final.

<?php

/**
 * @dataProvider provideConcreteSourceClasses
 *
 * @param string $className
 */
public function test_all_classes_are_final(string $className)
{
 $rc = new \ReflectionClass($className);

 $this->assertTrue(
 $rc->isFinal(),
 sprintf('Source class "%s" should final.', $className)
 );
}

But what if we have classes that we want users to extend some of our non abstract classes? Lets create a filter for that.

<?php

 /**
 * This array contains all classes that are extension points.
 *
 * @var string[]
 */
private static $extensionPoints = [
 ExtenableClass::class,
 OtherExtendableClass::class,
];

/**
 * @dataProvider provideConcreteSourceClasses
 *
 * @param string $className
 */
public function test_all_classes_are_final(string $className)
{
 $rc = new \ReflectionClass($className);

 if (in_array($className, self::$extensionPoints)) {
 $this->addToAssertionCount(1);

 return;
 }

 $this->assertTrue(
 $rc->isFinal(),
 sprintf('Source class "%s" should final.', $className)
 );
}

But of course, it doesn’t end here, we now also want to check if the list of extension points is still valid

<?php

 /**
 * @dataProvider provideExtensionPoints
 *
 * @param string $className
 */
public function test_non_final_non_extension_list_is_valid(string $className)
{
 $rc = new \ReflectionClass($className);

 $this->assertTrue(
 !$rc->isFinal(),
 sprintf(
 'Source class "%s" an extension point and should not be made final.',
 $className
 )
 );
}

public function provideExtensionPoints(): array
{
 return array_map(
 static function ($item) {
 return [$item];
 },
 self::$extensionPoints
 );
}

These two tests now make sure that all our classes are final, except for a specific list, and we also make sure that the list is still valid.

Source classes must have unit tests

Now this is a funny one, as you are testing if your code has tests. This also makes it easy to spot if a big PR is missing unit tests for certain classes.

The following method will check if all source classes have corresponding unit tests, except those that are in a ’legacy’ list

<?php

/**
 * This array contains all classes that are not yet unit tested due to legacy reasons.
 * This list should never be added to, only removed from.
 *
 * @var string[]
 */
private static $nonTestedConcreteClasses = [
 NotTestedClass::class,
 OtherClassWithoutTests::class,
];
/**
 * @dataProvider provideConcreteSourceClasses
 *
 * @param string $className
 */
public function test_all_concrete_classes_have_tests(string $className)
{
 $testClass = preg_replace('/VendorNamespace/', 'VendorNamespace\\Test', $className, 1) . 'Test';

 if (\in_array($className, self::$nonTestedConcreteClasses)) {
 $this->assertFalse(class_exists($testClass),
 sprintf(
 'Class "%s" has a corresponding unit test "%s", and can be removed from the non tested class list',
 $className,
 $testClass
 )
 );

 $this->markTestSkipped(sprintf(
 'Class "%s" does not have a corresponding unit test yet, you can improve this by adding one',
 $className
 ));
 }

 $this->assertTrue(class_exists($testClass),
 sprintf(
 'Class "%s" doesn\'t not have a corresponding unit test "%s", please add one',
 $className,
 $testClass
 )
 );
}

We use the dataprovider we created when making sure all classes are final, as we don’t usually have unit tests for interfaces or abstract classes.

Lets walk through the method and see what every part does, starting with the preg_replace.

$testClass = preg_replace('/VendorNamespace/', 'VendorNamespace\\Test', $className, 1) . 'Test';

Here, VendorNamespace is once again the ‘default’ namespace, as mentioned in Getting the classes. We manipulate the FQN we got to change VendorNamespace into VendorNamespace\\Test, and then append Test to it. For a lot of projects, this is how tests are named, simply the same class name, but with Test appended to it. We use preg_replace, with a 1 as the 3rd parameter, to make sure we only do this once, on the first occurrence. As your VendorNamespace may appear elsewhere within the namespace.

<?php
if (\in_array($className, self::$nonTestedConcreteClasses)) {
 $this->assertFalse(class_exists($testClass),
 sprintf(
 'Class "%s" has a corresponding unit test "%s", and can be removed from the non tested class list',
 $className,
 $testClass
 )
 );

 $this->markTestSkipped(sprintf(
 'Class "%s" does not have a corresponding unit test yet, you can improve this by adding one',
 $className
 ));
}

First we make sure that our list of not tested classes is still valid, by making asserting they are not tested, and otherwise notifying the user to update the list.

Next up, we mark the test as skipped. If you add verbose="true" to your phpunit.xml, or run it with the --verbose flag, all skipped tests will show their message. Meaning you will see exactly what source classes do not have unit tests every time you run those tests. This will make sure you always have your technical debt in sights.

<?php
$this->assertTrue(class_exists($testClass),
 sprintf(
 'Class "%s" doesn\'t not have a corresponding unit test "%s", please add one',
 $className,
 $testClass
 )
);

And to end it, we make sure that our classes in fact do have the unit test.

What’s next

First off, i would highly recommend checking out the AutoReview tests from PHP-CS-FIXER, where these tests originated. They were originally created by Dariusz Rumiński (keradus), who you should check out on github. By now other projects use these new types of tests as well.

For these checks, the options are almost endless, by using reflection, juggling some name spaces, and other methods, a lot can be tested before a PR has even been made. You could test classes don’t have public methods not coming from an interface, or make sure files like .travis.yml and appveyor.yml contain valid yaml. You can even check if the readme is up to date.

The most important part is that your error messages inform users why a test is suddenly failing, and what they can do to fix it, like adding another unit test, or making a property protected or private. And make sure that you cache what results you can when you run multiple tests against the same data set.

And last but not least, add an extra test set to make sure that your providers are valid. There are a lot of edge cases, so making sure that your lists of exceptions to the rules etc are valid, and that your providers/exception lists are valid helps a long way to reduce harder to debug error messages.

So go out there, and make your life a little bit easier by automating (part of) your code reviews!

The case against case

Mon, 18 Jun 2018 00:00:00 +0000

Chances are you have written a switch case statement or two. Aren’t they much ‘cleaner’ than a bunch of if else statements? Today i would like to convince you that using switch case, in modern php, is a bad practice.

Lets take a look at the following code snippet.

<?php
//test.php
declare(strict_types=1);

class One
{
 public function __toString()
 {
 return '1';
 }
}

function foo($value)
{
 switch ($value) {
 case '1':
 echo "got 1 \n";
 break;

 default:
 echo "did not get 1\n";
 break;
 }
}

foo(new One());
foo(1);
foo('1');

$ php test.php
got 1
got 1
got 1

As we can see, switch case checks if its value is equal rather than the same, it uses == instead of ===. This means switch case will juggle types, in order to see if the values it has are ’equal’. Even the declare(strict_types=1) doesn’t change anything about that.

Our code bases are filled with === checks, we use declare(strict_types=1) in our php 7+ projects. If possible we enforce both of those in our code standards. Yet, we still use switch and case, while very case is an equal check rather than a same check. Which means its a potential for bugs/ unexpected behavior.

So, if you are using switch case, refactor it into an if/else statement like so:

<?php
//test.php
declare(strict_types=1);

class One
{
 public function __toString()
 {
 return '1';
 }
}

function foo($value)
{
 if ($value === '1') {
 echo "got 1 \n";
 } else {
 echo "did not get 1\n";
 }
}

foo(new One());
foo(1);
foo('1');

$ php test.php
did not get 1
did not get 1
got 1

And suddenly, we are checking that the input is actually the string ‘1’, rather than something that is roughly the same. Of course, this is a rather simple example, but any switch case statement can be changed into an if else that will not type juggle. Another benefit of this is that magic methods like __toString() will not be called, just in case, god forbid, those have side effects.

Maybe if we are lucky we’ll see strict switch statements in future php versions. But until that day, lets stay away from switch case.

PHP extensions, polyfills and you

Tue, 12 Jun 2018 00:00:00 +0000

A little while ago someone ran into an error with a dependency of a project i worked on. A fatal error, which seemingly only occurred for them: the ctype_alnum function was not defined.

So it turned out the ctype functions aren’t part of the php core, but are instead a ‘default’ extension. Lets explore what extensions mean for your project, and how to help your users with these kinds of errors.

Extensions

You may have seen something like this in a composer.json:

{
 "require": {
 "ext-intl": "*"
 }
}

This means the project needs the intl php extension in order to work. If you don’t have it installed, composer will not install the packages, and tell you it is missing an extension it needs to function. Generally, installing a package is as easy as sudo apt-get install php-intl, and restart your web server if needed. A lot of deploy/build environments will automatically install the needed extensions, so no need to worry about that.

Even running composer already requires the simple-xml extension, so its not like you can run php without any extensions. A lot of packages require extensions without making it explicit like this, which is a shame. Getting an error when running composer install that mentions an extension being needed is a lot more informative than seeing the following error when running your program.

Fatal error: Uncaught Error: Call to undefined function Composer\XdebugHandler\ctype_alnum() in $dataRec->vendor/composer/xdebug-handler/src/XdebugHandler.php on line 437

A closer look at the reveals that from php 4.2 and onward, it is enabled by default. Meaning you would have to pass the --disable-ctype flag when compiling php to disable it. However, certain php distributions don’t include it, as they did in fact pass that flag while compiling, some examples being FreeBSD and Alpine. So relying on the fact that it is enabled by ‘default’ is not really an option.

There are of extensions available. Some are part of the ‘core’ meaning you can’t disable them, others are ‘default’, meaning a ’normal’ php installation should have it, but as mentioned early, we can’t rely on that. Some need to be installed by hand, and some even need a whole lot of configuration. So before you start requiring extensions left and right, it may be a good idea to check how much strain it will put on your users.

Polyfills

A way to avoid a direct dependency on (some) extensions is using a . Symfony provides quite a few, but there are others as well. So instead of requiring the ctype extension, we can require its polyfill. Composer can install that itself, so no error for the user, and if the user doesn’t have the extension available it will be ‘suggested’ by composer. Of course, this suggestion is buried by dozens of other suggestions that other packages make.

The one down side of using a polyfill is that it is generally slower. However, if you don’t have control over your production server, these polyfills mean you can still use the functionality they provide.

And you

So, what can you do with this information?

If you have an application for which performance is absolutely critical, consider checking what polyfills you are actually using, and what extensions are available on your production server. Even if you don’t use ctype or intl or another extension directly, your dependencies may use them, so getting the ‘real’ extension could give you a performance boost. This boost will be absolutely minimal, unless you use these functions a ton.

If you maintain an (open source) package, take a closer look at your code base, what extensions are you actually using, and are they presented in your composer.json ? The one line that says "ext-intl": "*" can save someone hours of debugging. Even when dealing with a ‘default’ extension, like ctype, it can an will be disabled on certain systems. If there is a polyfill available for that extension, you can also choose to require a polyfill instead.

You can also choose to work around any extensions that aren’t part of the core, but why would you? I’m not saying you should require every possible extension. Especially ones that are harder to install for your users, but most extensions should have better performance than the php implementation you could write yourself. But anything is better than requiring an extensions in your code, but not reflecting that in your composer.json

Test your tests are Testing

Sun, 10 Jun 2018 00:00:00 +0000

Are your tests testing? Are you assertions asserting?

Let’s find out how we can test that our tests are testing, with the mutation testing framework: infection.

Imagine this, you spend the last few hour refactoring a big part of the code base. You made some big changes and a lot of logic has been completely changed. Now it’s time to figure out what tests have broken due to all the refactoring. You open up the terminal and enter the good old vendor/bin/phpunit and watch the tests run.

$ vendor/bin/phpunit
PHPUnit 6.5.5 by Sebastian Bergmann and contributors.
Runtime: PHP 7.1.18
Configuration: /home/me/Projects/Important-Project/phpunit.xml
............................................................ 60 / 108 ( 55%)
................................................ 108 / 108 (100%)
Time: 24 ms, Memory: 4.00MB
OK (108 tests, 187 assertions)

That doesn’t look right, at least a few tests should have been broken by those changes. Some exceptions are no longer being thrown, return types have changed yet, your tests don’t care. According to them everything is still just as it was.

Thankfully, we have access to mutation testing, which will help you make sure that your tests are in fact, testing.

What is mutation testing

Have you ever created a unit test, and then changed something in your code to see if it failed? Well mutation testing is exactly that, but automated.

Let’s say you have the following class you are testing

<?php
class Calculator
{
 public function adds(int $a, int $b): int
 {
 return $a + $b;
 }

 public function subtracts(int $a, int $b): int
 {
 return $a + $b;
 }
}

And the following test to make sure it works

<?php
class CalculatorTest extends TestCase
{
 public function test_adds()
 {
 $calc = new Calculator();
 $this->assertTrue(is_int($calc->adds(1,2)));
 }

 public function test_subtracts()
 {
 $calc = new Calculator();
 $this->assertTrue(is_int($calc->subtracts(1,2)));
 }
}

We have 100% coverage, tests are green, so everything works as intended, right?

Of course, the subtracts method has a bug, instead of a - there is a +. A simple copy paste error, but our tests aren’t catching it.

So, how do we make sure our tests will catch those bugs? Through mutation testing. PHP has the mutation testing framework Infection, which is what we will be using for the example.

Let’s look at the interesting parts of the output:

4 mutations were generated:
 2 mutants were killed
 0 mutants were not covered by tests
 2 covered mutants were not detected
 0 errors were encountered
 0 time outs were encountered
Metrics:
Mutation Score Indicator (MSI): 50%
Mutation Code Coverage: 100%
Covered Code MSI: 50%

Four mutations were created, but only two were killed, while the other two escaped. Lets look at infection-log.txt to see what has happened:

Escaped mutants:
================

1) /home/gpagter/Projects/Randoms/calculator/src/Calculator.php:8 [M] Plus

--- Original
+++ New
@@ @@
 {
 public function adds(int $a, int $b) : int
 {
- return $a + $b;
+ return $a - $b;
 }
 public function subtracts(int $a, int $b) : int
 {


2) /home/gpagter/Projects/Randoms/calculator/src/Calculator.php:13 [M] Plus

--- Original
+++ New
@@ @@
 }
 public function subtracts(int $a, int $b) : int
 {
- return $a + $b;
+ return $a - $b;
 }
 }


Killed mutants:
===============

1) /home/gpagter/Projects/Randoms/calculator/src/Calculator.php:6 [M] PublicVisibility

--- Original
+++ New
@@ @@

 class Calculator
 {
- public function adds(int $a, int $b) : int
+ protected function adds(int $a, int $b) : int
 {
 return $a + $b;
 }


2) /home/gpagter/Projects/Randoms/calculator/src/Calculator.php:11 [M] PublicVisibility

--- Original
+++ New
@@ @@
 {
 return $a + $b;
 }
- public function subtracts(int $a, int $b) : int
+ protected function subtracts(int $a, int $b) : int
 {
 return $a + $b;
 }
 }

Let’s dissect what has happened, and see what it means for us.

When looking at the Killed mutants, we see two mutants, both PublicVisibility. What infection has done for these methods, is change their signature from public to protected. And then it ran all tests that are relevant for that line of code. The tests then failed, which means that the mutation was ‘killed’. Our tests detected this change in the code.

But what about the two ’escaped’ mutants? Our tests did not pick up on this change. When the + got changed to a -, our tests did not fail, all assertions were okay. This means that the mutant has ’escaped’. The tests did not pick up on the fact that our code changed.

If a test took too long, or caused a PHP error, it is also considered killed. The last option are mutants that are not covered by tests. Infection still takes these into account for metrics.

Lets go ahead and update our tests to kill these mutants.

<?php
class CalculatorTest extends TestCase
{
 public function test_adds()
 {
 $calc = new Calculator();
 $result = $calc->adds(1,2);
 $this->assertTrue(is_int($result));
 $this->assertSame(3, $result);
 }

 public function test_subtracts()
 {
 $calc = new Calculator();
 $result = $calc->subtracts(1,2);
 $this->assertTrue(is_int($result));
 $this->assertSame(-1, $result);
 }
}

Now we find that our subtracts method is actually returning 3 instead of -1. So we fix the bug, and run our tests again. They all pass, so lets run infection again as well.

4 mutations were generated:
 4 mutants were killed
 0 mutants were not covered by tests
 0 covered mutants were not detected
 0 errors were encountered
 0 time outs were encountered
Metrics:
Mutation Score Indicator (MSI): 100%
Mutation Code Coverage: 100%
Covered Code MSI: 100%

We did it, we fixed the bug, killed all mutants and saved the day!

Metrics

We have seen the Metrics in the infection output, but what does it mean?

Mutation Score Indicator (MSI)

Let’s say our code base has a total of 100 possible mutations. 40 of those are caught by the tests, either because the tests fail or because they cause a time out or a php error. 20 of them are not covered by tests, and another 40 don’t cause the tests to fail and ’escape’.

This would give us a MSI of 40% (The amount of killed mutations divided by the total possible amount.) You generally want this number to be close to your code coverage. If it is a lot lower it means your tests aren’t as good as you think they are.

It can be enforced, for example on CI, with the --min-msi flag.

Mutation Code Coverage

This the amount of mutations that are covered by tests, if we take the numbers of the MSI example, it would be 80%.

Covered Code MSI

This is the same as MSI, except only for covered code. So all mutations that are not covered by tests are ignored. If we take the numbers of the MSI example, we have a Covered Code MSI of 50%. You want this to be as close to 100% as possible.

It can be enforced, for example on CI, with the --min-covered-msi flag.

Usage

Installation

You can install infection in a few ways:

Through Composer as a dev dependency: composer require infection/infection --dev
Through Composer as a global package: composer global require infection/infection
Or as a phar:

$ wget https://github.com/infection/infection/releases/download/0.8.2/infection.phar
$ wget https://github.com/infection/infection/releases/download/0.8.2/infection.phar.pubkey

$ chmod +x infection.phar

If you want to update your phar you can do so by running $ ./infection.phar self-update.

Local usage

Running infection can take a lot of time if you have a big project or if your tests are slow. One way to speed it up is to run it with the following option: --threads=`nproc`. This will run the tests against mutations multi threaded, with as much threads as useful. Note: This should only be done if your tests can be ran parallel. If they use a database or the file system this can lead to a lot of mutations being killed that are not detected by your tests.

If you run infection during CI, its recommended to run it with the --min-msi and/or --min-covered-msi. This will force you to write better tests and keep your MSI and Covered Code MSI as a stable rate.

If you are already generation coverage on CI, you can hand infection that coverage, so it doesn’t have to generate it itself. Infection needs xml and junit coverage to run

$ vendor/bin/phpunit --coverage-xml coverage/coverage-xml --log-junit=coverage/phpunit.junit.xml

$ vendor/bin/infection --coverage=coverage

If you only want to run infection against changed code, you can run it like this:

INFECTION_FILTER=$(git diff ${GIT_PREVIOUS_SUCCESSFUL_COMMIT:-origin/master} $GIT_COMMIT --name-only | grep /src/ | paste -sd "," -)

$ vendor/bin/infection --threads=4 --min-msi=70 --only-covered --filter="${INFECTION_FILTER}" --ignore-msi-with-no-mutations

This will only run infection against changed files that differ from the last successful commit on this branch, or master if there is none, in the src folder. The --ignore-msi-with-no-mutations flag causes us to not error on min msi when we have 0 mutations.

For all the configuration options, you can check out infections documentation.