Mobile banking applications under scrutiny of Sii’s Testing Competency Center
Mobile banking applications are associated with comfort and the freedom of using the bank account wherever you are. We also expect them to meet the highest security standards. Are mobile banking applications really as comfortable and safe as we tend to see them?
How did the applications of the most popular banks in Poland do in tests carried out by Sii’s Testing Services Competency Center? Find out from the article which appeared on Bankier.pl „ Not all banks pass the tests of mobile applications” („Nie wszystkie banki zdają testy aplikacji mobilnych”). Accessibility (on how many devices can the application be used), performance and safety of access and data were some of the aspects tested by Sii specialists.
We present the full text of the article below.
NOT ALL BANKS PASS THE TESTS OF MOBILE APPLICATIONS
Mobile applications are a convenient and an increasingly popular way to access your bank account. However, not all organizations have satisfactory solutions in terms of performance, access authorization or data security.
During the 2017 I/O conference, Google has shared one of the most unexpected facts – the Android operating system is used on more than 2 billion active devices. Apple reported in January 2016 that a billion people use their devices on a monthly basis. By active devices we don’t mean phones only, but also tablets, fablets, etc. To these devices, one should also add other operating systems and a multitude of other devices – only last year Samsung released 31 new smartphone models. Huawei has released 22 new models, LG 19, Xiaomi 18, HTC 15, and Apple 3.
In this cross-section of processors, built-in memory, resolution, or architecture, it is difficult to find an optimal common denominator that ensures that all applications on each device behave the same way. It is obvious that on the most powerful devices applications run faster, smoother and look better, while older applications will freeze and loading times will be longer. Similar to mobile systems – different Android versions and different models will behave differently because each SDK version has its own properties, parameters and constraints. Adding to this the different models of one device, then the number of manufacturers and then the number of technical parameters – we get the number of instances that no programmer, no matter how good, will be able to predict and program. Meanwhile, application developers must not only optimize their programs, but also ensure security, product trust, and user experience – that is, what the user will receive as the final product – whether it is intuitive, well presented, etc. This is why it is very important to test mobile applications.
Hardware limits
Sii has taken several banking applications in its focus to look at their performance, intuitiveness, and security of operations. The applications of the most popular banks in Poland were tested. Starting at the deep end of the pool, we tested the availability of applications, i.e. how many devices can be used to test it. While in most cases a device limit does not exist, in some individual cases this limit was met and can be back breaking.
Some banks offered 1, 3 and 5 devices on which one could use the application. While 3 or 5 devices are still acceptable, it suffices when a bank application is used by merely two people. One device offered by one of the leading banks at present, has turned out to be a huge blunder. What if we share an account with our spouse, have more than one device, or simply use a tablet app instead of a smartphone? Such a restriction is, unfortunately, a shot in the knee, because it blocks the possibility of changing the device, and the changes in the system of the given bank –although possible, is not obvious and convenient for everyone. It requires waiting for a help center consultant , which will give us a PIN to enter the website (what if we are somewhere without a computer?) Or SMS codes. One can understand this procedure by the bank – security of data protection and prevention of leakage – this however, is very cumbersome and creates a huge inconvenience to customers.
Application performance
Not less important is the performance of mobile applications. The analysis in this area was conducted on the flagship and on the more popular phones – so that the tests are reliable and proven valid. More powerful and less powerful processors, different RAM sizes, several architectures were used. The test conditions were straight forward – no other application was running in the background, the device had access to Wi-Fi, it was fully charged, and all unnecessary services were disabled in the background. Applications were tested on various brands: LGE Nexus 5x, HTC One, Samsung Galaxy S3, Samsung Galaxy S5, Samsung Galaxy A3, iPhone 5s, iPhone 6 and iPhone 7 Plus. For testing, the SeeTestCloud tool by Experitest, the author of the study and with whom Sii collaborates, was used. This tool allows you to reliably measure RAM, CPU, battery usage, simulate various types of Internet connections, etc.
RAM and processor consumption was studied for logins, transaction history, transfer and logout, and the device status when logged out. One of the leaders in banking noted significant jumps on over-the-air operations, mostly on higher-end Android systems such as 7.1.1. This is a surprise because the device tested has 2 GB of RAM, allowing for a smooth and unobtrusive operation. The competition does not look any better – in one case the predecessor got weaker notes, there was a significant CPU and RAM jump for all operations on all devices. Can’t the developers of the application optimize the code or generate less background operations than is needed? The worst in this study was a foreign bank giant – on every device the application was jammed, noticeable was the CPU and RAM consumption, which at some moments reached almost 150 MB when clicking a button. Most applications had a low impact on performance, but from the moment of starting to the moment of killing it, generated high starting performance costs in the background. This does not have a positive effect on, among others, the battery life, so it’s worth a closer look. The application proved to be very contra-intuitive, it was difficult to find detailed information, filter the data, in order to learn navigation one had to pass the same path several times. The appearance itself is also quite skewed, because it contains a lot of gray icons which, in contrast to the red sum of payments, doesn’t look inviting. Unfortunately, in terms of performance and overall user experience, this bank application was the worst against the competition.
Access authorization and data security
Sii has also examined the authorization and therefore the security of our data, which we send around the world using mobile applications. In most cases, an 8-digit PIN is provided, or a login and a password – there are no cases that are far from this standard. The bank that was popular with small businesses was the worst in this competition. Only a 4-digit PIN was enough to log in, which can be easily remembered, however, it is a categorical security error. Such a low security combination on a bank application is unacceptable.
While most customers will not check security because they expect the application to be secure, Sii testers have looked at this topic closely. Elemental security errors were detected, which certainly does not encourage consumer confidence in the bank. Mobile applications must first and foremost securely communicate with bank servers and stay connected to them, which is a fundamental requirement when designing such an application, and apparently the mobile application developers have not foreseen this. Due to the confidentiality of the information, these errors will not be posted in the article, the full list of shortcomings will be sent to the bank to indicate security vulnerabilities.
Is it worth using mobile banking applications in spite of failures? Definitely, because applications are slowly becoming better and more efficient (though not all of them), and are not a direct and frequent target for hacker attacks (which is changing dynamically and should better not be easily dismissed), finally because this solution is most convenient. If we approach some of the products at a distance and reasonably judge our parameters, then we will see a lot of amenities. Let’s remember that the tests that were carried out were about what the app was and not what it offered.
Managers of bank IT projects should definitely consider increasing the number of mobile app tests – not just the elementary ones, but also regarding performance, security and functionality. As you can see, it is not hard to determine whether the application is written optimally. What we know is that users have older devices and older systems, because not all have flagship devices with a powerful processor or battery with the newest operation system.
Going with the times, banks are improving their offer to the extent which the latest technology will allow them to, so it’s important to monitor the quality of these improvements. Knowing from the tests conducted by Sii, not every bank has checked their products accordingly, and certainly did not check it for inconsistencies from other angles, which in the age of hacking or cybercrime can be a shot in one’s own knee. Approximately 20% of bank applications available in stores have a defect or error that could cause a catastrophic avalanche of unpleasant events as a consequence. So how can we prevent this?
Sii mobile applications tests
Sii, one of the IT leaders on the market, has the best Testing Competency Center for sofware testing. As the only company in the country, they work closely with Experitest, one of the world’s leading software testers. The fruit of this collaboration is the ever-growing range of mobile app testing and auditing services. Sii has a test lab – it’s a cloud of devices that is constantly being supplemented by more phones, not just the flagship ones, but also commonly used, slightly weaker models. This cross-section provides a thorough and above all reliable test of the application, as it can test the behavior and appearance of the application on multiple processors, architectures, resolutions and systems.
Entrusting basic performance testing to professionals provides not only measurable benefits, such as the comfort of releasing applications without defects, but also leads to higher financial profits. Additionally to the expense of a cloud device license cost, the cost of a single platform license (Android, iOS, Windows Phone, Blackberry, etc.) is added. For a company that wants to check one application, these costs are too high – it’s worth to optimize, so it’s much better to work with a company that has all the technologies already mastered and has already taken the burden of cost of buying all the necessary licenses. This advantage makes Sii with its offer very attractive and with this it outpaces the competition by leagues. Adding to the whole of the professional testing staff, who has a wealth of experience with various technologies and tools, we have the undisputed, best value proposition on the market.
Working with top-quality tools not only saves time, but also money. The SIA utility, SeeTestAutomation, allows you to create one script for multiple platforms, resulting in an up to three times faster performance than if you used a free or weaker tool. An additional advantage is the use of a network virtualization tool, so it is important to know how an application works with different types of Internet accesses, which is important today not only for user experience, but also for the security of packets sent to and from the application. Sii has a rich and diverse offer of tests, so consider collaborating with a company that has been ahead of the competition for years, has an experienced staff and is able to create flexible test plans for any application and for the most demanding clients.
