5 app vendor screening questions when outsourcing development

One of the most common questions we get asked as a software development agency in Hong Kong is “what differentiates you from other companies?” After we share our approaches and workflow in meetings, it is up to a client to decide which vendor to go with for their project.

We want clients to choose an appropriate app vendor for them, so we’ll share 5 quick questions that entrepreneurs and product managers can ask to compare dev shops for themselves. This is especially useful if you do not have your own in-house developer or designer to join the preliminary meeting with an app vendor.

Does your source code comes with unit test? If yes, what is your code coverage commitment?

Why ask: Good development team should have standard practices for unit testing. When asking, stress an understanding that this could be an additional cost so that the vendor does not become defensive in perceiving that you are trying to “squeeze” them or suggest that their code is not of high quality. Unit testing is a best practice for code QA. We have unit test code at backend and automated integration tests for APIs as a standard at Oursky.

Good Answers:
“Yes, and around 50 – 80% code coverage.” Note that there is no hard and fast “code coverage %”. It depends on the situation. In general lower than 50% is strange and more than 80% may mean that the tests are not very useful (as the test may match the code in a strange way).

Bad Answers:
“Yes, and around 100% (or 99%) code coverage.” This is bluffing, or a waste of time, likely because the tests are written in a way to cover all lines of code instead of considering usefulness.

“Yes, and we do it manually with another team” because unit tests are always automated and probably means the people answering probably are not actually familiar with unit tests.

Does your development team do regular code reviews? Can you share how it is done?

Why ask: A good development team should have another pair of eyes reviewing code regularly. It is difficult to review code beyond a few hundred lines. If a team does regular code reviews, their developers should have a habit of submitting code to another developer in small batches (approx 1 – 200 lines of code). These submissions are often called a “Pull Request”.

Good Answers:
“Yes, we do it on each pull request by another developer.” For example, a dedicated developer on a project may submit a pull request with a clear commit comment.

“Yes, we do it for important source code by another developer.” This may be the case if a vendor cannot afford the cost of reviewing each pull request.

Bad Answers:
“Yes, we review it every week / month / upon project completion.”

Intervals may seem logical, but they do not work for development workflow. This is likely bluffing. Unless it is code review for a security audit or some sort of knowledge share session / internal training, reviewing a lot of code at once is considered a bad practice. Code should be reviewed before being merged and then easily rolled back if there is a problem. There is simply too much code to review after a week’s worth of work and signals other problems in the development process of the company.

Does your design team read and follow the iOS Human Interface Guidelines and Material Design Guidelines for Android?

Why ask: Mobile app design is very different from graphic design or general visual design. This question finds out whether a designer is equipped with a basic understanding of how UX/UI design works in mobile. The design guidelines that Apple and Google created for iOS and Android respectively are regularly updated.

This should be a simple “Yes / No” question.

To further verify, have your team’s designer join the meeting so that they can ask and talk more about best practices with the vendor’s designer. Ask their point of view after the meeting.

How many junior / senior developers does your team have? How do you maintain quality across teams with different compositions?

Why ask: No vendor can afford not to have some junior (0 – 5 years usually) developers in their team. In addition, not investing in junior developers is not healthy for the software engineering ecosystem in the long-term, so a healthy ratio is expected from any good development team that invests in training junior software engineers.

A good development team should have different measures to ensure quality of work. This could include code review by a senior technical lead for each pull request, regular technical sharing, standardization of frameworks, proper technical planning, and/or a mentorship or buddy system.

If I give you a completed UX/UI Design, how would you like me to provide the design requirement to your developers?

Why ask: This question wants to learn two things:

  1. if a vendor’s design team has a standardized output for developers, and
  2. if their developers have a standard way to work with designers.

This question helps find out if the vendor has a design process. Alternatively, you can also ask if the vendor has an in-house development team and how your design team can handover the design to the developers.

Good Answers:
You can provide us the original design files (e.g. Sketch / Adobe XD CC / Figma / Photoshop etc), with design specifications and design guidelines. (Here’s our design process.)

Can you send us a {insert any design specification software, such as Zeplin}?

A good design and software development team will also be able to give feedback where design specifications or guidelines may be incomplete and have a checklist of items needed.

The reason they will ask is because these design specifications become the blueprint for developers to build the product, and they are ensuring that everyone is on the same page. If development begins without complete UX/UI designs, there could be issues ranging from missing features to software architecture that did not account for important use cases.

The summary is that when asking your questions, pay attention to how an app vendor gives a response as much as what their response is. A good design and development team will be able to give realistic answers and reasonable explanations.

Rather than look for the vendor who gives the highest or most ideal numbers, dig deeper into how they work and make their estimations (we use EBS). Though you are outsourcing your product development, these designers and developers are your partners and they need to be able to share with you from day one how they work transparently in order to build a successful digital product in the long term.

If you have an idea for a product or are looking for a code review for your existing product, get in touch!

One Reply to “5 app vendor screening questions when outsourcing development”

  1. Pingback: Antonette Lundsten

Leave a Reply