BYTE the Cloud

Chris 0:00
All right, welcome everyone. We're diving into AWS Device Farm today. It's a service that's well, it's becoming more and more important for cloud engineers, especially,

Kelly 0:07
yeah, especially for those who are working with mobile apps, definitely, exactly.

Chris 0:11
Yeah. Imagine you just finished developing like, the next big app, you know. But how do you make sure it works, like on all the different devices out there, all the different phones and tablets, 1000s of them, yeah, 1000s so many different operating systems, screen sizes. It's

Kelly 0:29
a nightmare. It

Chris 0:29
is. It really is. That's where AWS Device Farm comes in, exactly.

Unknown Speaker 0:33
That's exactly the problem to tackle. So

Chris 0:35
instead of, you know, buying and managing like a whole lab of devices,

Kelly 0:39
oh, which can be a huge pain and so expensive.

Chris 0:43
Oh, tell me about it. So you're saying with Device Farm, you can just run your tests on like a ton of real devices that AWS hosts for you.

Kelly 0:51
That's it. Think of it as like a giant virtual test lab right at your fingertips. Okay,

Chris 0:56
so it's like having that giant lab, but without all the hassle and expense, exactly,

Kelly 1:01
and you get access to a way wider range of devices than well most companies could afford to buy on their own.

Chris 1:08
That makes a lot of sense. So for those who might not be super familiar with it, could you just give us a quick definition of AWS Device Farm, like, what is it exactly, and why should cloud engineers care about it?

Kelly 1:19
Absolutely So AWS Device Farm, in a nutshell, it's a cloud based service, right? And it lets you test your apps, web and mobile apps on real

Chris 1:29
physical devices, real devices, not emulators or simulators, nope,

Kelly 1:33
real devices. And that's why it's so important, because you need to make sure your app works perfectly across different operating systems, different screen sizes, all the different hardware variations out there. You're saying that's crucial for user experience. It's absolutely critical and ultimately for your app's success too. And

Chris 1:49
you're saying emulators just can't really replicate that. Can they? They can't.

Kelly 1:52
Emulators are fine for initial development and testing, but they just don't capture those little quirks and nuances of real world devices the way the hardware and software actually interact Exactly. So Device Farm gives you the most accurate results possible because it's testing on those actual devices. I can

Chris 2:09
see how that would be a huge advantage. You can catch those device specific bugs before they you know, before they reach your users

Kelly 2:15
Exactly. Save yourself a lot of headaches later on. Okay, so

Chris 2:19
I get the idea. But can you give us some real world examples, like, how are companies actually using Device Farm out there in the wild? Sure,

Kelly 2:27
think about a gaming company, right? They're about to launch this awesome new mobile game, lots of graphics, very demanding, right? They need to make sure that game runs smoothly on everything from the latest, you know, high end phones, flagships. Yeah, flagships down to the older, more budget friendly models with less

Chris 2:47
processing power, less memory,

Kelly 2:49
exactly. Device Farm lets them test on all those different devices and make sure every user has a good experience. Okay,

Chris 2:56
that makes a lot of sense for gaming. What about, I don't know, what about other industries like health care maybe, sure.

Kelly 3:03
Perfect example, think about a health care provider. They've got this mobile app for their patients,

Chris 3:08
for appointments, maybe, or to access medical records, right, exactly.

Kelly 3:11
So they need to be absolutely sure that app is working perfectly on all sorts of devices and operating systems, reliable and secure, absolutely especially if it's handling sensitive medical data. You know, you're talking

Chris 3:24
about hip agree, compliance and all that, exactly, Device Farm

Kelly 3:27
can help them with all of that. It helps ensure reliability and and helps them maintain that compliance to protect patient data and, you know, build trust.

Chris 3:36
So we've established that Device Farm solves a real problem, a big pain point, especially for cloud engineers working with these mobile apps, right? I want to get into the nuts and bolts of the service itself. Okay, sure, what are some of the key features and benefits that that our listeners should really know about, what makes it tick? Well, one

Kelly 3:55
of the biggest advantages is just the sheer number of devices they have available. Oh yeah, yeah. I mean, we're talking hundreds of real Android and iOS devices. Wow, covering all kinds of brands, models, screen sizes, operating system versions, you name it.

Chris 4:10
So you're saying it's like a way broader selection than most organizations could ever put together on their

Kelly 4:16
own. Exactly. It would just be too expensive, I

Chris 4:18
can imagine, and I'm guessing it probably cuts down on the time and resources needed for testing too, right? Absolutely, another

Kelly 4:25
big feature, parallel testing. Parallel testing, yeah, you can run your tests at the same time on multiple devices. Hello. Okay, so instead of like, manually testing the same thing on 50 different devices, one by one, right, which would take forever, exactly, device firm does it all at once in a fraction of the time. Okay, that's

Chris 4:43
a huge time saver. Yeah. What about the results? Like, what happens when a test fails? How do you know it went wrong?

Kelly 4:50
Oh, device firm gives you really detailed reports. Oh, you get logs, screenshots, even videos of the test executions, videos so

Chris 4:59
you can actually. See what happened during the test exactly. You can pinpoint exactly where and why a test failed, and that makes debugging a lot easier, way easier.

Kelly 5:07
It's like having, you know, X-Ray vision into your app. You can see exactly what's going on.

Chris 5:11
And it's not just about, you know, identifying failures, right? Yeah, you're also getting performance data. Absolutely.

Kelly 5:18
You can see how your app performs on different devices under different conditions, so you can optimize it, optimize

Chris 5:24
it to make sure it's running smoothly across the board, Exactly Okay. So let's see. We've talked about the wide range of devices, the parallel testing, the detailed reports, the performance insights. Sounds like Device Farm offers a pretty comprehensive testing environment. It does. But how does it? How does it fit in with the rest of AWS, you know, the whole ecosystem of services.

Kelly 5:48
It's very well integrated, actually, okay, works seamlessly with S3 for example, for storing stuff, yeah, you can store your app binaries, there, test scripts, test results, so everything stays within AWS. Nice and tidy, exactly. It also works with Lambda. Lambda for automation. You got it. You can automate your testing workflows like you could set up a Lambda function to trigger tests automatically whenever, let's say, you deploy a new build of your app. Oh, that's pretty cool. Or if code gets pushed to a certain branch in your Git repository, Right exactly. So it ties in with, you know, modern CICD pipelines very tightly. That's awesome. Okay, so far so good. But, you know, it can't all be perfect, right? Are there any limitations, anything our listeners should be aware of?

Chris 6:35
Well, you're right. There are always trade offs, like they do have a massive selection of devices, right? But they might not have absolutely every single device out there, especially some of the really obscure ones,

Kelly 6:47
yeah, or some of the really old models that nobody uses anymore, exactly. And of course, there's always the cost factor, right? You have to think about cost, even though Device Farm probably saves you money compared to, you know, building your own device lab? Oh, yeah, absolutely. But the costs can still add up depending on how much you use it, especially if

Chris 7:05
you're running a ton of tests on a lot of different devices, yeah.

Kelly 7:08
So you really need to weigh the benefits against the cost, right, exactly, and figure out the best testing strategy for your specific needs. Okay,

Chris 7:16
that makes sense. Speaking of making informed decisions, I know a lot of our listeners are also prepping for their AWS certification exams, right? So let's shift gears a little bit, and let's dive into some example questions, the kind of things you might see on the exam. Okay, good idea. You know, related to Device Farm. Let's do it. I'm actually kind of curious to see how I do here. All right, hit me with your first question. Give you your best shot. All

Kelly 7:37
right, let's see. So imagine you are working on a mobile app that needs to be tested on a variety of Android devices, different versions of the operating system your team is using, CICD, and they need to automate the testing process. Okay, so which AWS service would be the best fit for this scenario? Hmm.

Chris 7:56
Okay, let me think sounds familiar. We were just talking about a service that lets you test on real Android devices, right, supports different operating system versions and and it can be integrated into a CICD pipeline using a Lambda, using Lambda, right? So that would be able Device Farm, you got it.

Kelly 8:17
Device Farm, that's the one. It checks all the boxes real Android devices, different OS versions, automation with Lambda, perfect fit for a CICD pipeline.

Chris 8:26
Okay, yeah, I see why that's the obvious choice. Yeah, that was a pretty, pretty straightforward question, right? The warm up, yes, yeah, all right, let's try a harder one. What about Okay? What about push notifications? Okay, you know, those are super important for a lot of apps, especially for user engagement, absolutely. So let's say I'm building an app that, you know, really relies on those push notifications to keep users coming back, right? I want to make sure those notifications are working properly. You know, they're actually being delivered, they're displaying correctly all sorts of devices, right? Yeah, on all sorts of devices, including, like, you know, older models that might be running older operating systems. Those can be tricky. They can Yeah, so how can I use Device Farm to test that whole push notification flow? That's

Kelly 9:09
a great question, and it really highlights why testing on real devices is so important. Simulators, they often just they can't really replicate that accurately the whole push notification thing, right? Because

Chris 9:20
it's not just about the app itself. It's about how the device's operating system handles those notifications

Kelly 9:26
Exactly. So with Device Farm, you can actually write tests that specifically target those push notifications. Oh, wait, those tests, they send notifications to the devices in your test pool, and then they verify a bunch of stuff, like, what first did the notification actually get to the device? Okay, yeah, that's important. Then, is the notification showing the right content, and when the user interacts with it, does it trigger the right actions in your app?

Chris 9:52
So you're testing the whole end to end flow, basically from the server sending the notification all the way to the user, actually seeing it, interacting with. That on their phone

Kelly 10:00
exactly, and all of this is happening on real devices, not some, you know, idealized simulation. So you

Chris 10:06
could be pretty confident that your notifications are actually going to work out there in the real world, on all kinds of different devices. That's the idea. That's the goal. That's really cool. Okay, okay, so we've talked about push your notifications, right? What about user authentication? Okay, like, let's say my app requires users to log in, right? A lot of apps do, yeah. Most do, yeah. How do I handle that? With Device Farm? Do I really have to, like, manually log in on every single device in my test pool? No

Kelly 10:36
way. You don't have to do that. Oh, good. That would be a nightmare, especially if you're testing on hundreds of devices. Yeah, it would be impossible. So Device Farm gives you a couple of options for handling authentication. One way is to use built in testing frameworks. And those frameworks, they let you script the whole login process. Oh, okay, interesting. So it happens automatically on each device. So I

Chris 10:58
could, like, write a script that enters the username, the password, even like two factor authentication codes if I need to exactly,

Kelly 11:05
saves you a ton of time and makes sure the login process is consistent across all your test devices.

Chris 11:11
That's really smart. Are there other ways to do it? Yeah, you could also use

Kelly 11:15
pre authenticated test builds of your app. Okay,

Chris 11:18
so in that case, the app would already be like logged in when the tests start running, exactly,

Kelly 11:22
yeah. So you can bypass that whole login step. Okay, that makes sense, and just jump right into testing the features that require a logged in user.

Chris 11:30
Cool, it seems like Device Farm really has thought of everything. They've

Kelly 11:34
tried to make it as easy as possible to handle all these common testing scenarios. That's

Chris 11:39
great. Okay, let's go back to those exam questions for a minute. Are

Kelly 11:43
you ready for another one hit me? Okay, so let's say you're testing your mobile app on Device Farm, and you notice that it's performing really poorly on one particular device model. You're seeing high CPU usage. The app's getting sluggish, maybe even unresponsive. Oh, that's never good. No, it's not, yeah. So how would you use Device Farm to investigate and diagnose this issue?

Chris 12:06
Okay, so this is a performance problem. Yeah, I need to figure out what's going on. But I can't, like, physically hold the device in my hand and, you know, poke around inside, right? So what can I do? Well, this

Kelly 12:17
is where those detailed reports that Device Farm provides. They become really valuable. Now you get all these performance logs, graphs showing CPU usage, memory consumption, even network traffic logs, all

Chris 12:29
specific to that device, Right exactly?

Kelly 12:31
So you can pinpoint the exact moment when the CPU usage spikes, you know, on that problematic device.

Chris 12:37
And then I can try to correlate those spikes with like other things that are happening in the app, maybe figure out what's causing the bottleneck

Kelly 12:44
Exactly. And don't forget, Device Farm also captures screenshots and videos during the test runs. Oh,

Chris 12:50
right, so I can actually see what's happening on the screen at the same time as the CPUs going crazy, exactly.

Kelly 12:55
Really helps you understand what the user's seeing and what might be causing that performance issue.

Chris 13:02
It's like a detective tool for mobile app testing. You

Kelly 13:04
got it. It really helps you see what's going on under the hood. Okay, that's super helpful.

Chris 13:08
What about managing access to Device Farm?

Kelly 13:13
Okay? You

Chris 13:14
know, in a team environment, yeah,

Kelly 13:15
that's important. Let's

Chris 13:16
say I've got a whole bunch of developers all working on different parts of the app, right? How do I make sure that they only have access to the devices and the resources that they actually need for their specific tests? We don't want anyone accidentally breaking something or messing up someone else's testing environment.

Kelly 13:32
That's a great point, and that's where I am. Comes in. I am identity and access management. That's the one so Device Farm integrates really well with IAM, which means you can control who has access to what with a lot of granularity.

Chris 13:46
Okay, so it's just like managing permissions for other AWS services exactly.

Kelly 13:49
You can create specific IAM roles for your developers, and you can limit their access to, let's say, certain device pools or projects,

Chris 13:58
because they can't, like, accidentally launch tests on devices they shouldn't be using

Kelly 14:01
exactly. You can even control what actions they can perform, like whether they

Chris 14:05
can create new projects or modify existing ones, or even just view test results for projects they're not supposed to be working

Kelly 14:12
on exactly. It's all about making sure you have a secure and controlled testing environment.

Chris 14:18
Makes sense. That's important. Okay, I've got another exam style question for you. Okay, bring it on. All right. So I'm using Device Farm to test my app, but I'm finding that the tests are taking a long time to complete, okay, like, way too long,

Kelly 14:31
right? Happens sometimes, yeah,

Chris 14:32
it's really slowing me down. So how can I speed things up without, you know, without sacrificing test coverage. I don't want to cut corners. That's

Kelly 14:42
the challenge, right speed and thoroughness, exactly. So

Chris 14:45
what are my options? Well, first thing I would do is take a look at your tests themselves, okay, like make sure the tests are actually efficient, exactly.

Kelly 14:52
Are they well structured? Are they only testing what they need to test? Are there any unnecessary steps? Or you. Redundant checks.

Chris 15:00
So basically, optimize the test code itself exactly.

Kelly 15:03
You can often shave off a lot of time just by cleaning up your tests and making them more efficient. Okay,

Chris 15:09
that makes sense. And I know Device Farm supports parallel testing, right? Yes, absolutely. So I could run tests on multiple devices simultaneously. That's one of the biggest advantages of Device Farm, yeah. And that would definitely help reduce the overall testing time, big time.

Kelly 15:23
And you have a lot of control over how you parallelize those tests. Oh, yeah, yeah. You can choose to run different tests on different devices, or you can run the same test on a bunch of devices at the same time. Okay, that's really flexible. It is. Give you a lot of options. Cool,

Chris 15:39
anything else they could do to speed things up? Well,

Kelly 15:42
you could try using faster device types for your tests. Oh, right. Newer devices, more powerful processors, more memory, they'll generally execute the tests more quickly.

Chris 15:51
So instead of testing on, like, a huge range of devices, I could just focus on a smaller set of, you know, more modern devices, especially

Kelly 15:59
for those parts of your app where performance is really critical, yeah? Okay, that's

Chris 16:03
a good strategy. Of course, you

Kelly 16:05
still want to test on a variety of devices for compatibility, but if you're really trying to optimize for speed, yeah, focusing on those faster devices can help. That makes sense.

Chris 16:16
Any other tips you can share, like anything else I should be thinking about, well,

Kelly 16:20
if you really need to get the absolute fastest testing times, and if your budget allows for it, okay, you might want to consider using a dedicated device pool. A dedicated device pool, what's that? So with a dedicated device pool, you get exclusive access to a set of devices, okay, which means your tests aren't competing with anyone else's tests for resources.

Chris 16:39
Oh, I see. So it's like having my own, like, private testing lane with no traffic jams,

Kelly 16:44
exactly. It guarantees that your tests will run as fast as possible without being slowed down by, you know, other users or anything like that.

Chris 16:53
Okay, that's pretty cool, but probably more expensive too, right? Yeah, it

Kelly 16:56
does cost more. It's something to consider if speed is absolutely critical,

Chris 17:00
okay, that's good to know. All right, one last question about Device Farm before we wrap up this whole exam prep section. Okay, what about what about integrations with other AWS services? We talked about S3 and Lambda earlier, right? But are there other ways that Device Farm can connect with the rest of the AWS ecosystem? Oh,

Kelly 17:19
absolutely. Device Farm is very well integrated with a whole bunch of other AWS services. Okay, good. Like, what? Give me some examples. Well, we already mentioned S3 for storage and Lambda for automation, but there's also CloudWatch.

Chris 17:33
CloudWatch that's usually used for monitoring, like servers and applications, right?

Kelly 17:38
But you can also use it to monitor your Device Farm tests. Oh, interesting. You can set up alarms that notify you if tests are failing or if they're taking too long to complete.

Chris 17:47
So it's a way to stay on top of things, you know, be proactive about any issues that might come up, exactly. And then there's code pipeline. Code pipeline for CICD, yep. So you can actually integrate Device Farm right into your code pipeline workflows so I could, like, automatically trigger a bunch of tests on Device Farm every time I push new code to my repository. Exactly.

Kelly 18:07
It's all about making testing a seamless part of your development process. That's

Chris 18:11
really powerful. It's not just about test 10 and isolation. It's about integrating it with everything else you're doing. Exactly, okay? What else?

Kelly 18:19
What other integrations Am I missing? There's also SNS, SNS for notifications. Yep, you can have Device Farm send you notifications about all sorts of things, like what test completions, failures, any event you want to be alerted about.

Chris 18:33
So I don't have to, like, constantly be checking the Device Farm console to see what's going

Kelly 18:37
on exactly. It just keeps you in the loop automatically. Okay,

Chris 18:40
so it seems like Device Farm really does play well with others. It does.

Kelly 18:44
It's not just a standalone service. It's designed to be part of a larger ecosystem. And that's really what makes AWS so powerful, right? Absolutely, all these services working together, it's a game changer. Okay,

Chris 18:55
I think we've covered a lot of ground here, I think. So I'm feeling pretty good about Device Farm. Now. You're good, that's the goal. I'm ready to take that exam.

Unknown Speaker 19:01
I think you are. You've got this all right. But

Chris 19:03
before we, before we totally wrap up this deep dive, I have one last question for you. Okay, what is it I have been thinking about the future of mobile app testing. Okay, you know, where do you see Device Farm fitting in, like, five years from now, 10 years from now, what's next? That's

Kelly 19:21
a great question. I mean, mobile technology is changing so fast is Yeah, it's hard to predict, but I think, I think testing is only going to become more important. Yeah, that makes sense. We're already seeing these more complex app experiences, you know, augmented reality, virtual reality, right? And the whole Internet of Things, thing, exactly. All that stuff is getting integrated into mobile apps. So the devices themselves

Chris 19:43
are changing too. Oh yeah, for sure. I mean, foldable phones are already a thing, right? Wearables are becoming more and more popular. What's next? Right? Exactly, yeah. So how does Device Farm keep up with all that? Well,

Kelly 19:56
AWS is constantly updating it. Okay? Yeah, adding new devices, new features, new integrations. So

Chris 20:03
it's not just a static service, it's actually evolving. Oh, yeah, it has to keep pace with the industry. Exactly.

Kelly 20:08
I think we're also gonna see a lot more automation. Okay, interesting. And AI powered testing, AI

Chris 20:16
like machine learning. Yeah, exactly. Imagine

Kelly 20:19
using machine learning to, like, analyze your test results, maybe even predict potential issues before they happen. Wow, that would be amazing. It would and maybe even suggest ways to optimize your

Chris 20:31
app. That's kind of like taking things to a whole new level. It

Kelly 20:34
is. So you know, the skills that you're learning today with Device Farm, they're going to be even more valuable in the future. Okay, that's

Chris 20:40
good to know. I'm ready for the challenge. I think you are. All right. Well, this has been an awesome Deep Dive. It has been fun. I feel like I learned a ton Good. I'm glad I'm ready to go build some amazing apps and test them thoroughly on Device Farm. There

Kelly 20:53
you go. That's the spirit. And remember, there's a whole community out there, you know, tons of resources, online forums, meetups, people I can connect

Chris 21:02
with, ask questions exactly. Yeah, cool. Well, to all our listeners out there, thanks for joining us for this deep dive into AWS Device Farm.

Unknown Speaker 21:10
Yes, thank you.

Chris 21:12
We hope you learned a lot, and we hope you're feeling inspired to level up your mobile app testing game.

Kelly 21:18
That's what we're here for. Keep

Chris 21:19
learning, keep experimenting, keep building great stuff, absolutely and until next time, happy, coding. Everyone. Bye.