Recertification Reprise: AWS Certified Machine Learning - Specialty

I have to admit… after my last near disaster experience with remote proctored testing, I was especially on edge.  In fact, I’m not even supposed to be here! My journey with machine learning is probably not common, but as I’m finding, it doesn’t seem to be that unique either.

You see, I’m not a “math person”.  This confusion and frustration with higher-level mathematics is what I credit for getting me into computers in the first place.  Rather than drudge through my math homework manually, it was far more “efficient” for me to whip together a few lines of BASIC code on my Radio Shack TRS-80 Color Computer II.  

In college things didn’t improve for me.  I failed Calculus – twice – which effectively changed the trajectory of my career. (I also failed FORTRAN but that’s a story for another time.) 

Despite my mathematical shortcomings, I’ve been able to bumble through life as a technologist fairly successfully.  Little did I know, my Math Monster would come back around to haunt me…

Ready to get started on your AWS Machine Learning journey? Get started with an ACG free trial today to access courses, quizzes, hands-on labs and practice exams to set yourself up for success.

• My background
  • Down the ML rabbit hole I go
  • Enter the heresy
• The cert exam
  • My proctored exam experience
  • A (second) near-disaster
• My cert exam strategy
  • The first pass
  • Second and subsequent passes
• My preparation
  • General areas to cover
  • Using practice exams
  • Making things stick
• My advice
  • Be aware of the “Practitioner’s Curse“
  • Notice what AWS considers important
  • Don’t sweat the new stuff
  • Get those hands dirty

I consider myself a professional learner. I’m curious and always looking to dig into new things – especially into areas that seem to be full of hype and outpaced expectations.  Around half a decade or so ago, machine learning certainly fit this description.  

I had to know what this ML stuff really was capable of. And more importantly… could I harness its powers to outsmart the stock market?

Like many others wanting to learn something new, I enlisted the assistance of our benevolent search engine overlord, Google.  Almost immediately, I found plenty of courses aimed at Machine Learning beginners and selected one to give it a go.  Well, ten minutes into the course, the instructor started in with….yep..calculus.  My old arch-nemesis!

In fact, most other so-called “beginner“ machine learning courses I tried also had this similar theme.  The instructors presumed their audience naturally thought in terms of equations and greek symbols with the same fluency they themselves did.  

(By the way, this was an important lesson for me long before I got into the teaching and training business. Assuming your audience is just like you is a very dangerous assumption.)  

The instructors were and still are very well-regarded in the machine learning space with impeccable credentials. But try as I might, it just didn’t make sense to me.

In my two machine learning courses, on social platforms, and in talks I give, I’m vocal about my opinion that you don’t need to be a math genius to understand how various machine learning methods work. Some have taken issue with this perspective. Perhaps they think I’m being clickbaity or trying to sell some get-certified-quick scheme.

Nothing could be farther from the truth. In fact, if I had a deep mathematics background, my learning curve would have been much shorter.

Instead, I had to work harder and dig deeper for metaphors and practical examples and explanations to understand things that the math-talkers might have easily taken for granted.

Over the years, I’ve managed to amass enough machine learning knowledge through one-off tutorials, self-taught exercises and slogging through academic-level papers on the subject.  I am certainly not a machine learning ‘expert’; hire me to design a new deep neural network approach for pharmaceutical lab trials at your own peril!  However, I do know enough about how various machine learning approaches and algorithms work, and most importantly, I have an idea of the practical limitations of various machine learning approaches.

I first took the AWS Certified Machine Learning exam in beta form at re:Invent 2018, shortly after it was first released. Amid the hustle and bustle that is re:Invent, I went into that exam unprepared, rushed, and not feeling confident at all. 

I received my results when the beta closed and was not surprised that I had failed with a score of 704 (750 is passing). However, that experience gave me a much better understanding of what the exam covers and the level of detail. After brushing up on some of my weaker areas, I sat for the exam again in April 2019 – this time feeling much more confident and prepared.

I passed handily.

Fast forward three years and here I am again, ready to recertify.  

I opted for PSI as my provider and remote proctoring out of familiarity with the process. After my prior near-disaster of my laptop somehow failing to take advantage of the charging cable, I wasn’t going to take any chances this time. Hardwired Mac Mini, hardwired keyboard, hardwired mouse and hardwired ethernet cable. Some VLAN and QOS magic on the home router to ensure plenty of bandwidth. (Peppa Pig’s not getting any of MY exam bandwidth!)

I was ready.  Bring it on!

The PSI exam launch process was nearly identical to my last exam recertification in October.  After downloading the exam client, it performed a hardware and internet connection check and soon enough, I was greeted by a proctor via chat. I had to pan my webcam around and under my testing area and show my arms and ears clearly to ensure I wasn’t concealing any listening devices or cheat sheets. Satisfied, the proctor then released me into the exam.  The concept of some anonymous person watching you as you take the exam might be a little creepy but I soon forgot about it and just focused on my task.

I can confirm that someone is actually watching those video feeds because I did unintentionally raise the ire of the proctor once. I have a habit of resting my chin on my fist when reading off a computer screen (the pose would be reminiscent of those 90’s yearbook photos with the red and blue laser backgrounds).

Apparently, at one point during my exam, my knuckle slightly obscured my mouth… which is verboten per the remote proctoring rules. (Perhaps the concern was that with my mouth obscured, I could have been narrating the question into a hidden recording device and exfiltrating confidential question data.)  

Within seconds after I shifted my resting think position, the proctor popped up in the chat window to remind me not to obscure my mouth.  Of course I complied and carried on with my exam.

Time management is always important when taking timed exams (says Mr. Obvious). There are different techniques you can employ, but my strategy has served me well. I always plan on doing at least three or four passes through the exam. Let’s break it down:

For my first pass, I read all the questions and answers, then mark my best guess. If I’m sure of the answer, I move right on along. If I have the least little bit of hesitation, I will flag the question for a future pass. My plan is to complete the first pass through all the questions with about half the time remaining.  

(Hot tip: For AWS exams, I will always answer questions even if it is a guess as there are no penalties for incorrect guesses.)

With my first pass complete, I will return to the flagged items, re-reading the question and answers with more care. 

Sometimes, this re-read will be enough to identify some key piece of information that I didn’t catch in the first round. And, like clouds parting to reveal blue sky, the correct answer immediately leaps off the screen for me. 

At other times, I might still be drawing a blank, so I’ll leave it flagged and move on.  As I continue to do pass after pass, the number of flagged items starts to dwindle. Using this multiple-pass approach, I avoid getting bogged down in the tough questions that pop up early on.

Every time I revisit a flagged question, I read it as if it were totally new and intentionally leave any preconceptions behind. This helps me keep an open mind and notice things that I might have missed in prior reads. 

I can recall one question in particular where this strategy helped me out. I had subconsciously inserted an assumption into the scenario which caused me to waffle between two answers. It took two more passes before I noticed the question very obviously dispelled my assumption, which easily left me with only one plausible answer.

As the blueprint hadn’t changed since the last time I took the exam in 2019, I knew that I was in for generally the same type of questions. Do be aware that the exam covers much more than just Sagemaker. Sure, the SageMaker family of services is pretty vast but you should also figure in adjacent services such as AWS Glue, S3, Athena, VPCs and such. (Yes, VPCs on a machine learning exam… you won’t be asked detailed questions about VPC transit gateway routing, but you might be asked about ensuring the security of your SageMaker instances within VPCs.  Similarly, you won’t see Redshift database design questions, but you should know what functionality Redshift ML can provide.)

General AWS architecture knowledge will also help you out. If you have a good understanding of when to use S3 versus EFS versus FSx for Lustre with regard to the costs and performance tradeoffs, you’ll be in a good place for those questions around data storage and handling.  Conceptually, deploying SageMaker inference endpoints is very much like a load balancing deployment you might do using normal EC2 instances. A canary deployment or blue-green deployments are very much relevant for SageMaker and map over conceptually.

With those more general items under my belt, I used the practice exam on the ACG platform as a diagnostic. As I don’t work with SageMaker every day, I had forgotten some things and the practice exam helped refresh my memory. Granted, I had written some of the questions in that question bank so I could vaguely recall the particular trick or objective that I had in mind, but it was still a good exercise.

Through my preparation, when I encountered a concept or term that I was a bit fuzzy on, I added it to a list for deeper research. Things that always give me the most trouble are – not surprisingly – statistics concepts like L1/L2 regularization, softmax, and normalization vs standardization.  I made sure to spend extra time on those items.

I am embarrassed to admit this but confusion matrices have always… confused me. I get true negatives, true positives, false negatives and false positives mixed up when trying to apply them to some problem description. This then starts to mess me up when it comes to the associated metrics of recall, precision and F-1 score.

This time around, I decided to fix this confusion once and for all. 

I created a comic strip. 

Just putting these things in a visual form with an associated practical story involving a robot at a vitamin sorting factory finally helped me commit this to memory. Maybe I’ll release the comic some day as a study tool… 

Want to understand the AWS Certification landscape? Check out our popular AWS Certification Guide to know the ins-and-outs of AWS certs.

You’d think that someone with years and real-world experience in data science and machine learning might have a definitive advantage on this exam over a newcomer, but use caution here. Real-world experience allows us to learn all the things that you don’t usually learn about in formal training programs. As a practitioner, you’ve likely faced challenges and problems that have enabled you to build that experience and intuitive knowledge about various situations. But this intuition can sometimes be a disadvantage on certification exams like these.

Those of us who have been around for a while will unconsciously build in certain assumptions into use cases based on our experiences. These assumptions can influence how we understand the exam questions and might lead to choosing incorrect answers. Like it or not, these exam questions live in the walled gardens of documentation. The more experience you have, the more difficult it becomes to leave all your proverbial “baggage” at the figurative door of your testing center.

Pay attention to what AWS considers noteworthy. I’ve found the areas in AWS documentation highlighted as “Note” are particularly fertile ground for exam questions. You don’t need to memorize these things but do be sure to read over them when browsing the documentation.

There are new features coming out under the AWS ML umbrella almost every week. But don’t worry about those too much for your exam prep. Per AWS, services or products must be generally available for at least six months before appearing on a certification exam. If some new things do show up, it’s likely an unscored beta question.

At least in the current rendition, the machine learning exam does not have any hands-on components.  But one of the best ways to reinforce your study is to actually do hands-on stuff. (And I’m not just talking about simply skipping through Jupyter notebooks pressing Shift-Enter!)  

If you need some inspiration, consider some of these ideas:

• Build some actual models and deploy some inference endpoints and put an API Gateway in front of it.  Call it using PostMan and see if you can get them to respond properly.
• Create a second model and deploy it alongside the first such that it gets a portion of the requests.  
• Take on some of the hands-on labs on machine learning in the ACG library.
• Try the machine learning Cloud Guru Challenge hosted by Kesha Williams.
• Use some data from the Registry of Open Data on AWS to index with AWS Glue, query with Athena and visualize with QuickSight.
• And my favorite, learn about hyperparameters in action by training an AWS DeepRacer model!

Even if you don’t have a deep mathematics background, you can understand machine learning and pass this exam. I’m living proof!

You can also subscribe to A Cloud Guru on YouTube for weekly cloud news, like us on Facebook, follow us on Twitter, and join the conversation on Discord.