Friday, January 27, 2017

AUTOMATION 28 - Some checks are faster than others ... surviving The Pyramid Of Doom

This is part 28 of my automation series - for our story to date, check out the list of articles here.

Technical level: **

All the way back in part 4, I talked about a zoology of automation types.  My lingo has evolved a bit, and I sometimes talk about "the automation spectrum" or yesterday I talked about "the automation toolbox".  Different terms for the same thing.  😉

There are three basic types of automation we've been talking about to date,

  • Unit testing, which occurs on parts of the code itself.  It's really great for checking business rules.
  • API testing, which involves the integration usually of an application layer to a lower layer, but can be used to drive a front end as well.  It works by inserting commands and scenarios for a system to react.
  • GUI testing, which is testing from the user interface.  Entering data, pushing buttons etc.


A decade ago when we were talking about automation within the context of testing, we were almost always thinking about a GUI test tool.  Typically the notoriously unreliable record-and-playback tools.

To really be smart in their use, it helps to not just choose one, but to learn to use them all.

Why?  What's the advantage?



One of the key things is speed!

Try using this online dice roller here.  When you click to roll the dice it,

  • sends a request back to the server
  • the server rolls a dice
  • the server renders a new page
  • your browser has to load up a new page (I know because I'm on a slow connection and can see it loading)
  • the page has to download an image corresponding to the number (it takes a moment for this to load too)
  • done

All told it can take about 2 seconds for a roll.  There is another site which is as slow, and there's an animation which shows a rolling dice, so it really can't go faster.

When we were looking at dice rolling under our unit test, we were able to run about 600,000 dice rolls in the same time.  All that test needed to do was to call the method which generated the random number.  So the round-trip is considerably faster (it helps it doesn't have to travel half way around the world and back for sure).

This is generally what we find in automation - unit testing is several orders of magnitude faster than GUI testing, with API testing somewhere in-between.



In my consultation with different automators, a great piece of advice they've given is all around thinking about testing at the appropriate layer.  This was summed up by a quote "don't test the database with a GUI tool".  A bit like the online dice roller, you have to take a long trip through a lot of layers, which you need to do once in a while, but if there's a lot of test you need to do, you want to move them much closer to the area you're testing.

An example came out last week when we were talking about ways to test a csv file upload.  I showed a method I used, we have line-by-line error messaging for uploads, so I make a file full of friendly uploads.  Then I make one where the first line is good, but line by line I break a rule - whether too many characters here, or a number in a text only field etc.

It makes for a great manual test, and we were thinking about how to automate it, originally as a GUI test.  Then we realised that each line where I'd kept or broke the rules really belonged as a JUnit test - it'd be much faster that way.

Something else which came out when we discussed this further at our company technical test study group was there was another reason to push for more JUnit testing.  It's also far easier to write a JUnit test for complicated data than it is to write a GUI one,

  • In a GUI test you tend to have to create a data item going through a series of processes.  For instance if you needed a suspended customer account, you might have to create it, validate it, then go in as an admin and suspend.
  • In a JUnit test you can just declare an item, and have the ability inside the code to set certain fields.  For instance, "declare a customer object, and set the status to suspended".


This makes for much simpler tests, but it brings challenges.  Developers and testers need to be able to look more at what's in place in terms of unit tests, and to make things visible.  We tend to have lovely dashboard for status of GUI and API tests, but the JUnit checks can be a little less visible to a tester (although you'd hope if a test fails, it won't build, so it'd be always 100% right?).

Beware The Pyramid Of Doom! *

The idea that to take best use of the speed of unit and API tests (which also tend to be more robust as well because data and interface items tend to change much less than GUI elements) you need more of them than GUI tests has been represented in a model you've probably heard of called the automation pyramid which I first read about in Lisa Crispin and Janet Gregories Agile Testing.


Simply put, because they're so much faster and easier, you'll want more unit tests than API tests than GUI tests.

Hopefully you'll see from the above why - faster and simpler!

Lisa and Janet's book was written in 2009, but as we discussed in our study group, go back just a few years around New Zealand, and we pretty much had a test pyramid then ... it just went the other way ...


As we've embraced more agile methodology and particularly the idea of automation as a toolbox over a tool, we're taking huge leaps to embrace this pyramid more in our use of automation.





* Unless you haven't noticed, The Pyramid Of Doom in automation looks somewhat like this ...


13 comments:

  1. Saya ingin berbagi cerita kepada anda bahwa saya IBU HERAWATI seorang TKW dari malaysia dan secara tidak sengaja saya buka internet dan saya melihat komentar IBU DARNA yg dari singapur tentan AKI SYHE MAULANA yg telah membantu dia menjadi sukses dan akhirnya saya juga mencoba menghubungi beliau dan alhamdulillah beliau mau membantu saya untuk memberikan nomor Togel/lottrey toto 4D dr hasil ritual/ghaib melalui jarak jauh dan alhamdulillah itu betul-betul terbukti tembus dan menang RM.230.000 Ringgit ,kini saya kembali indon membeli rumah dan kereta walaupun sy Cuma pembantu rumah tanggah di selangor malaysia, sy sangat berterimakasih kepada AKI SYHE MAULANA dan tidak lupa mengucap syukur kepada ALLAH karna melalui AKI MAULANA saya juga sudah bisa sesukses ini. Jadi kawan2 yg dalam kesusahan jg pernah putus asah, kalau sudah waktunya tuhan pasti kasi jalan asal anda mau berusaha, ini adalah kisah nyata dari seorang TKW, AKI MAULANA adalah guru spiritual terkenal di indonesia. jika anda ingin seperti saya kunjungi situs/website KLIK DISINI RITUAL PESUGIHAN DUNIA GHAIB



    SUCCESS STORY OF HERAWATI MOTHER

    I want to share the story to you that I am HERAWATI MOTHER a TKW from malaysia and accidentally I open internet and I see DARNA MOTHER comment from singapur about AKI SYHE MAULANA who has helped him become successful and finally I also tried to contact him and alhamdulillah he Want to help me to give the number Togel/lottrey toto 4D dr ritual results / unseen and alhamdulillah it really proved translucent and won RM.230.000 Ringgit, now I am back indon buying house and train although sy Just a housemaid at selangor malaysia, sy very thankful To AKI SYHE MAULANA and do not forget to give thanks to ALLAH because through AKI MAULANA I also can be as successful as this.So kawan2 who in distress jg ever break down, when it's time god must be the road of origin you want to try, this is the true story of a migrant worker, AKI MAULANA is a famous spiritual teacher in Indonesia. If you want to like me visit the website CLICK HERE RITUAL GHAIB TEMBUS LOTTREY


    ReplyDelete
  2. Saya ingin berbagi cerita kepada anda bahwa saya IBU HERAWATI seorang TKW dari malaysia dan secara tidak sengaja saya buka internet dan saya melihat komentar IBU DARNA yg dari singapur tentan AKI SYHE MAULANA yg telah membantu dia menjadi sukses dan akhirnya saya juga mencoba menghubungi beliau dan alhamdulillah beliau mau membantu saya untuk memberikan nomor Togel/lottrey toto 4D dr hasil ritual/ghaib melalui jarak jauh dan alhamdulillah itu betul-betul terbukti tembus dan menang RM.230.000 Ringgit ,kini saya kembali indon membeli rumah dan kereta walaupun sy Cuma pembantu rumah tanggah di selangor malaysia, sy sangat berterimakasih kepada AKI SYHE MAULANA dan tidak lupa mengucap syukur kepada ALLAH karna melalui AKI MAULANA saya juga sudah bisa sesukses ini. Jadi kawan2 yg dalam kesusahan jg pernah putus asah, kalau sudah waktunya tuhan pasti kasi jalan asal anda mau berusaha, ini adalah kisah nyata dari seorang TKW, AKI MAULANA adalah guru spiritual terkenal di indonesia. jika anda ingin seperti saya kunjungi situs/website KLIK DISINI RITUAL PESUGIHAN DUNIA GHAIB



    SUCCESS STORY OF HERAWATI MOTHER

    I want to share the story to you that I am HERAWATI MOTHER a TKW from malaysia and accidentally I open internet and I see DARNA MOTHER comment from singapur about AKI SYHE MAULANA who has helped him become successful and finally I also tried to contact him and alhamdulillah he Want to help me to give the number Togel/lottrey toto 4D dr ritual results / unseen and alhamdulillah it really proved translucent and won RM.230.000 Ringgit, now I am back indon buying house and train although sy Just a housemaid at selangor malaysia, sy very thankful To AKI SYHE MAULANA and do not forget to give thanks to ALLAH because through AKI MAULANA I also can be as successful as this.So kawan2 who in distress jg ever break down, when it's time god must be the road of origin you want to try, this is the true story of a migrant worker, AKI MAULANA is a famous spiritual teacher in Indonesia. If you want to like me visit the website CLICK HERE RITUAL GHAIB TEMBUS LOTTREY


    ReplyDelete
  3. Selamat Datang di Blog Nagasaon Datu Bolon Izinkan kami membantu anda semua dengan Angka ritual Kami Kami dengan bantuan Supranatural Bisa menghasilkan Angka Ritual Yang Sangat Mengagumkan Bisa Menerawang Angka Yang Bakal Keluar Untuk Toto Singapore Maupun Hongkong Kami bekerja tiada henti Untuk Bisa menembus Angka yang bakal Keluar..dengan Jaminan 100% gol / Tembus Tapi Ingat Kami Hanya Memberikan Angka Ritual Kami Hanya Kepada Anda Yang Benar-benar dengan sangat Membutuhkan Angka Ritual Kami Kunci Kami Anda Harus OPTIMIS Angka Bakal Tembus Hanya dengan Sebuah Optimis Anda bisa Menang…!!! Apakah anda Termasuk dalam Kategori Ini 1. Di Lilit Hutang 2. Selalu kalah Dalam Bermain Togel 3. Barang berharga Anda udah Habis Buat Judi Togel 4. Anda Udah ke mana 2 tapi tidak menghasilkan Solusi yang tepat Jangan Anda Putus Asa Anda udah berada Di blog yang sangat Tepat Kami akan membantu anda semua dengan Angka Ritual Kami Anda Cukup Mengganti Biaya Ritual Angka Nya SajaJika anda Membutuhkan Angka Ghoib Hasil Ritual Dari DATU BOLON 2D,3D,4D di jamin Tembus 100% Silahkan Hub : Eyang Guru 0852-3162-7257 Adapun Besar Pergantian Biayanya Tergantung Paket Yang Kami Berikan 1. PAKET 2D : Rp.300.000 2. PAKET 3D : Rp.500.000 3. PAKET 4D : Rp.700.000 Semua Biaya Diatas Tidak Sebanding Dengan Angka Yang Kami Berikan Kepada Anda…Tapi Ingat Setelah Anda Succes Dengan Angka Ritual Kami..Sisikan Sedikit Buat Orang Yang Membutuhkan Nya Dan Untuk Bisa Mendapatkan semua Angka Kami Silahkan Anda Selasaikan Paket Yang Ada Diatas Call / Di Nomer : 0852-3162-7257 EYANG GURU – DATU BOLON FORMAT PENDAFTARAN INFO BOCORAN SGP & HK; Kpd Yth!! Bpk/ibu Yang Sedang Dililit Utang Kami Atas Nama Team Togel Yang Terbaik Bersedia Ingin Membantu Anda Dgn Angka GOIB, Hasil RITUAL Sendiri, Cara Bergabung Jadi Member ? KETIK: Nama Anda#Kota Anda#Kabupaten#Togel SGP/HKG# LALU Kirim Ke No Hp Kami : ( 0852-3162-7257

    ReplyDelete
  4. I want to share the story to you that I am a Web Service Provider, Online Casino Play Baccarat Online Casino Via Mobile Phone, IPhone, IPad and Roulette Slots...etc "


    จีคลับ
    goldenslot
    gclub online casino


    PLZ, come & join with us, Thanks for your many tutorials!

    ReplyDelete
  5. Terimakasih banyak AKI karna melalui jalan togel ini saya sekarang sudah bisa melunasi semua hutang2 orang tua saya bahkan saya juga sudah punya warung makan sendiri hi itu semua berkat bantuan AKI JAYA yang telah membarikan angka 4D nya menang 275 jt kepada saya dan ALHAMDULILLAH berhasil,kini saya sangat bangga pada diri saya sendiri karna melalui jalan togel ini saya sudah bisa membahagiakan orang tua saya..jika anda ingin sukses seperti saya hubungi no hp O85-244-015-689 AKI JAYA,angka ritual AKI JAYA meman selalu tepat dan terbukti..silahkan anda buktikan sendiri. 2D 3D 4D 5D 6D








    Terimakasih banyak AKI karna melalui jalan togel ini saya sekarang sudah bisa melunasi semua hutang2 orang tua saya bahkan saya juga sudah punya warung makan sendiri hi itu semua berkat bantuan AKI JAYA yang telah membarikan angka 4D nya menang 275 jt kepada saya dan ALHAMDULILLAH berhasil,kini saya sangat bangga pada diri saya sendiri karna melalui jalan togel ini saya sudah bisa membahagiakan orang tua saya..jika anda ingin sukses seperti saya hubungi no hp O85-244-015-689 AKI JAYA,angka ritual AKI JAYA meman selalu tepat dan terbukti..silahkan anda buktikan sendiri. 2D 3D 4D 5D 6D








    Terimakasih banyak AKI karna melalui jalan togel ini saya sekarang sudah bisa melunasi semua hutang2 orang tua saya bahkan saya juga sudah punya warung makan sendiri hi itu semua berkat bantuan AKI JAYA yang telah membarikan angka 4D nya menang 275 jt kepada saya dan ALHAMDULILLAH berhasil,kini saya sangat bangga pada diri saya sendiri karna melalui jalan togel ini saya sudah bisa membahagiakan orang tua saya..jika anda ingin sukses seperti saya hubungi no hp O85-244-015-689 AKI JAYA,angka ritual AKI JAYA meman selalu tepat dan terbukti..silahkan anda buktikan sendiri. 2D 3D 4D 5D 6D



    ReplyDelete
  6. Thanks for this post Queader! Now I've got to get on and implement more thank you.

    gclub online
    goldenslot
    สูตรบาคาร่า

    ReplyDelete
  7. Looking for
    best selenium training in Chennai, Credo Systemz is the no 1 selenium Training institute in Chennai offering professional selenium course by selenium experts.

    ReplyDelete
  8. The risk is superior.
    จีคลับ The option to open online casino games that will bring you like stepping into a real casino. Many online gambling games are ready to offer the best possible bets and meet every need. With a new generation of online games designed for today's gamers. The number of gamblers who choose to gamble online. A lot of glee of gambling. We are a great online service provider that is ready to give you full enjoyment. Bet on gambling online, bet fully on all kinds of bets. Lots of fun that we have brought a lot of games to you to bet on the mobile phone screen. Get the most out of your site, and get instant access to our site and expertise. Only players have the Internet, they risk the chance to win online every time. We offer fast and agile online betting. It is convenient to gamble on all online gambling games. If you do not want to miss out on the great online gambler that offers its users the money to profit from the game. We provide you with unlimited bets and entertainment to players at any time. บาคาร่า

    ReplyDelete
  9. Entertainment gives you 24 hours with online casinos.
    Ruby888It is another online casino games site that can provide you with a real risky online betting experience. Online horoscope has been broadcast online from the famous gambling source of Cambodia. Casino Poipet Casino Resort Enjoy the fun and full of fun to win. Because of the new type of gambling games, both the easy understanding. The challenge is to deliver on the experience of gambling. Guarantee the fun of a long-established website as well as the recognition of the player who bets. We are the leading online gambling site in Thailand. Realistically open to players all day and all night. There is a team of experts who provide advice. Ask for opinions and ask questions. About Betting Games Anytime Online gambling online that will give players a chance to challenge the game. Moreover, the money spent on the service game. All games are ready to make you rich. The opportunity of online casino gambling that most users wait to use the service every click. IBCbet

    ReplyDelete
  10. Real Play Free Play
    Gclub SlotBetting on Online Casinos is a great way to earn money online. Many new online casinos that will make you gambling. With the kind of risky online games that play comfortably. The opportunity of gambling to win online gambling. With hundreds of online games to choose from. Online gambling games are popular among gamblers around the world. This is because they can bet on the full scale and earn money from the games. The atmosphere of online casino games is more realistic by live casino games online. With sharp images, play without interruption. Giving the best online gamble to the consumer. Not only that, they can find tips and entertainment with other buyers. Entertainment is not a holiday to open the risky online games that meet the needs of customers. If you are interested or are looking for ways to make extra money to customers. Win a game online is a way to choose which can meet all users needs. Holiday Palace

    ReplyDelete
  11. Online gambling has better entertainment.
    Goldclub SlotIt is a great way to start playing online gambling games at the best online gambling sites. Win a chance to win no limit online. The customer can risk the online fortune of our website whenever you want. Online gambling betting opportunities suitable for all types of gamblers. Gold to the joy. Every game is a gamble to be sure that the players are ready to bet. And can meet the needs of online gambling games are open all the time. And online gambling games where you choose to gamble. Online casino games more than 100 games to bring joy. Excited to you constantly. Online casino games and players will be more than entertainment. We offer online casino services that are all so popular and popular. Do not miss the chance to win live online and also promote. Special offer for new members and get up to 500 free credits to gamble. All bets are free to play. คาสิโนออนไลน์

    ReplyDelete
  12. Think of the service at once. Remember to remember our online casino.
    viva9988 Our online casino site. It's a fast-paced web site hosting service. And the speed of live broadcast through the Internet, covering all locations, unlimited. No matter where you are We have developed programs and systems all the time to accommodate more and more modern technologies and to meet the wishes of the players. With no play. You can play on your mobile just by having the internet. No need to waste time downloading the program. In addition, each game is a virtual game developed from casinos in foreign countries. Please join us by signing up to become part of our website. I guarantee that. You will not be bored anymore. And your game is not a normal game. That's your dream. Unless you are having fun then. You also have the opportunity to win prizes from participating games. Come back home We have a Jackpot to distribute each day. From various games You will only be eligible if you join our site. If you are interested in joining us at our website. จีคลับ

    ReplyDelete
  13. The blog are the best that is extremely useful to keep.

    ดูหนัง

    ReplyDelete