?

Log in

No account? Create an account
UI Automation by Mocking
anime
he_the_great

The testing pyramid is a popular communication tool to show where the quantity of testing should occur. Some people have to complain about the analogy. In essences the push is to have a reduction in tests which happen at the UI and with full system integration (end-to-end testing). There is plenty of information out there on why you want to do that, like the article I linked above.

While I haven't actually made this path happen, I'm looking at yet another layer of separation to help facilitate the UI testing, mock the backend interactions. The goal here is to identify the primary state changes the UI is expected to under go, verify the UI requests to the backend and to return events back to the UI to validate its behavior to that response. This keeps the end-to-end part of the setup to be less complete, though a good mocking of backend system is still an undertaking.

The other aspect to UI testing is that Selenium is quite popular, but isn't really stable or consistent. So I'm also going to focus on utilizing only headless Chrome for the majority of UI testing. Once that testing is complete I can look at what is left to test with the end-to-end environment.

Tags:

When is it OK to Find a Bug
anime
he_the_great

"How many times have you seen an email praising the heroic efforts of the developer who fixed some last-minute major issues in a huge new feature that were discovered in final user acceptance testing? This should not be seen as a heroic deed—rather it should be seen as a tragic failure. " — Real World DevOps

Let's be clear, finding issues is always best before it exists. However, we have layers of testing specifically because we want to find issues. The fact that you found the issue during testing is a good thing and a success.

Quick side note on User Acceptance Testing, at this point I believe that the tests should be focusing on, now that the system is in place rather than testing that features meet business requirements, you want to be testing that the system as a whole is meeting the value/proposition expected. The types of issues I hope to find here aren't ones where the behavior deviates from the specified requirements, but that the requirements were not satisfactory with either what was actually expected or that expectations changed due to other factors.

The two main points I want to note is

  • Reproducing the issue quickly in lower environments
  • Getting the fix to the environment quickly

If either of these things don't happen, then you have a failure. These eliminate the "heroic" efforts because it reduces time taken to fix a bug.

Read more...Collapse )

Why Communicating with Git Helped You Version
anime
he_the_great

In my previous postings about Git I talked about communication, and now that you've been communicating well we can get into what that means from a strategy standpoint.

Versioning is a large topic area and I do favor the idea of supporting several versions, but encourage limitations on that and discourage indefinite version support. But I want to focus on the idea of supporting the development versions: production, in flight, and new work.

New work is simple, you're preparing for the next release and you want to get everything you can into it.

The trouble comes when you're wanting to now make a change to production or an in flight release. If it is production the change should make it into all three, if it is in flight then it should go to new work also. Generally you will want to make the change to the oldest version first and cherry-pick and port into the later versions, but that is not always the case. Sometimes the bug you identified during new work becomes a desired fix in those older versions. Since you've been diligent about your communication you should now have a issue number and specific changes all ready to go and place into the older versions.

QA will be much happier if you're utilizing the original changes stored in your version history than recreating the changes through copy/paste or other manual means. You will also be happy if the changes are more complex than a few line changes.

Read more...Collapse )

Git and Other Communication Tools
anime
he_the_great

My last post on Git tried to focus on an end goal for your history. But I can already hear the nagging reply's "We have other tools which we use to communicate these things."

Most likely you're utilizing some form of bug tracking system, these systems help you to organize what is changing and communicate back what has been completed. These tools are very important as they provide external input, records of future work needed, and facilitate planning efforts. They can also facilitate pretty graphs and development cycles.

In my view, these tools make the reality of clearly communicating changes in your commit history even more important. In all likelihood your git commits can be integrated with your bug tracking system, not only to leave comments on the issue/bug but to cause resolution/closure of them.

If your read my last post, I referenced that during work on one feature some additional bug may be identified and fixed. When you break this fix out into its own commit, you'll start to wonder, "What issue number do I reference in this commit?" The answer to this question will likely result in, "I need a new issue number to assign this commit to." Otherwise if you're doing code reviews, the reviewer will look at your commits and ask, "Where is the record this work was done, I don't see an issue number assigned?"

By communicating we have better checks and balances and expectations on what and where we communicate.


Git is a Communication Tool
anime
he_the_great


Historically there has been a number of complaints on the way git allows people to change history. There are continued discussions on using rebase, merge, and/or squash. I want to bring focus to the point of all these choices, communicating. I'm not looking at providing new suggestions, someone has already written about why you should do one thing or another.

Who might you want to be communicating with? 

  • Future self
  • Other Developers
  • QA member
  • Project Management

What is it you might want to communicate?

  • That a feature is complete (to the best of your ability at this time)
  • This code change is needed to fix bug X
  • I have this important documentation update
  • I was doing some work and my editor decided to change these files and I don't know why.
  • I don't like the formatting of this document, here is my recommendation
  • I completed X, Y, or Z and think it is important to get these upstream for everyone to benefit.
  • Can we please upgrade our visual studio project files to the latest version of .net?

Now as a developer, wouldn't it be great if this is actually how you developed your work? Nice clear goals, good discipline in only doing one thing, ignoring that bug you found because it isn't the task at hand? Well, no it wouldn't be good you'll forget about that bug, or you will be so busy documenting that it needs fixed you forgot where you were in your task, it's just much better to get those changes into the code base and move on. And this is why Git's history rewrite is so valuable.

Read more...Collapse )

Full Registration
anime
he_the_great

One thing I've come to realize in software testing is to never entertain the idea of a full regression. The idea behind a full regression is to collect a set of test which cover the software's functionality to be confident it can perform the defined behaviors.

The problem comes from expectations, that all test ever run will be run again, after all we said full didn't we. In an agile environment with mid-sprint, when to run this full regression also becomes a question.

Personally I'm also leaning away from this concept of defined regression testing, at least outside automated verification. Following changes and targeted exploritory testing should be preferred.

Tags: ,

Don't Reinvent the Wheel
anime
he_the_great

In software development this phrase, "Don't reinvent the wheel" is thrown out as an important mantra to follow. The main problem is that this gets thrown out there without looking at the wheel market. Clearly who ever said this doesn't make use of wheels. There are innovative wheels all over the place.

Do you drive a truck, car, tractor, or a wheel barrow? Guess what, someone determined that the old wheel wasn't good enough for those and invented a new one.

Just take a look at the Wikipedia page on the phrase. It is less about building the same thing and more about building something new with an understanding of the old.


Future Of Testing Recap
anime
he_the_great

As a software tester, I started looking advice/resources and came across talks by James Bach. And he has really resonated with me. Other blogs and information I find lacking. I wasn't looking for tutorials on how to utilize tools, I wanted testing methodology and communication I wanted more of what James talks about in Future of the Software Testing Role. This is my opinionated summary of this talk, some of it is direct quotes some is my interpretation others are my closest approximation to what he said, but it all indicates the general time for when he said it:

2:45 — At this point James is setting the stage of where testing has been going and what has been driving it. He references the Agile movement and how this is a process set by developers and not testers, which is why you don't see testing as part of the process but just a side effect of the team doing work.

4:00 — He expresses that Agile's testing approach and his own are both in response to bad testing/testers. He is looking to improve the quality of the craft while feels Agile wants it gone.

4:30 — At this point James starts to make the case for specialization, but does really like the idea from Agile that people take on different roles, everyone should be testing. He later expands on this (1:07:00) saying he would like to see roles more like a villa.

Read more...Collapse )
Tags: ,

Continuous Delivery
anime
he_the_great

I get the feeling that I'm doing something wrong. For whatever reason I keep trying to write little utility programs to make this continuous delivery stuff work. I'm going to specifically call out to GitLab and partly Jenkins, but expect it to apply to most tooling.

See, GitLab CI Runner allows for use of shell scripts, docker and such. Jenkins allows for Groovy scripts. But really that isn't what I want these little helper tools to utilize. The primary tool I'm thinking of is a program which sets my application version based on some git describe information.

Right now I'm installing these tools directly to the build server (in the case of Jenkins I could use the copy project artifacts). These tools are store in their own repository and used by the build scripts, so how come my CD system doesn't have a way for me to continuously deploy my helper programs to the system it uses? 



Graphical Interface Editing
anime
he_the_great

I started doing some work with the WPF and the xaml editor. I don't really like GUI editing and certainly haven't found the right framework to do it in. But I've always been partial to editing the graphical source code rather than utilizing a WYSIWYG.

I just wanted to make the comment that WPF seems to have brought back editing the backend code rather then rely on the editor. I just wish it wasn't XML.