Journey to enhancing my AI
Hello once again! In today’s article I’ll be giving a report/review on my process to improving the accuracy of my AI from totally awful to not so totally awful — not much of a difference but a difference still!
I decided to make this review a bit more fun and delivered like a story, enjoy!
P.S. It will be retold in third person
1: The problem
After her recent successful recreation of the AI to be used for the purpose of weed identification Steph knew her adventure wasn’t yet over — she couldn’t just stop there.
The AI could have been a lot better with many more opportunities to implement various improvements and she knew it. Therefore she put it upon herself as a challenge to overcome and got ready to embark into a new adventure . She was very excited for the journey ahead but that was only because she lacked a thorough understanding of the perils to be faced…
2: The plan
Steph went to seek advice from her trusted mentor Esther and she definitely wasn’t wrong in doing so. The meeting with her mentor yielded more fruits than expected and helped Steph to clearly see the path she would take to succeed in her new mission.
So what was this path? Steph aimed to complete her mission within 2 weeks with subtasks in between to get closer and closer to her overall aim which went beyond improving the accuracy of the AI but into embedding it with an app so as for it to be more interactive.
3: The journey
So the journey began in the wild west amidst the sand and foreign dunes — and cut. None of that was serious, so let’s restart.
Where the journey actually began for Steph was by looking into reorganising her data and actually going through it cleaning it and setting it out in all the required folders (train, val, and test folders). She believed that once her data was organised and improved she could then just run the previous model again and test it out to get more accurate results, she was far from right.
What actually occurred is that once she got all the data set and organised properly the true colours of her model’s disappointing inaccuracy was unveiled from its seemingly perfect mask.
A previous accuracy of 80% went down to 20% and despite it being bad at least there was transparency within that as an 80% accuracy would not get test images so widely wrong the 20% accuracy was more fitting to explain the AI’s constant failures.
She did not let this setback stop her progress instead she took it as an obvious sign to change her approach. She sought out help from others around her who were quite knowledgeable in the building of an AI and eagerly awaited their suggestions…however those did not come in at good timing and were not as responsive as desired. So her next best place to seek answers was obviously Google search and Perplexity. She quickly found a suitable explanation for her constant failing with her AI the conclusion was that a CNN structure just wasn’t suitable for an AI with the sole purpose to identify objects through visualisation so she decided to try out a YOLO structure for a change whilst hoping for the best.
Some other suggestions from her network came underway soon after but she had already made up her mind on switching to YOLO so she didn’t bother delving too much into other possible solutions, and immediately dived into figuring out how YOLO worked.
What you now see above is what Steph used as her holy guide through the stormy paths of the YOLO land which at the time was unknown to her. One could argue she fell in love with the channel which so clearly and pleasantly guided her step by step through every different part of the preparation for the creation of a proper YOLO structure from the data annotations to the structuring of the the folders and the making of the yaml file (a file used to make the model aware of the actual structure of the data indicating to various important information such as the classes , the name of the classes , their locations in the device etc..).
Despite the very clear guidance that did not promise a smooth road quite the opposite in fact — the thing about tutorials is that everything works out perfectly in them but when you attempt recreating it something must go wrong(don’t debate me on this). Steph had many moments where she felt like giving up and seriously questioned her abilities to achieve success of any form. The only thing that kept her going through those hard times was the ever so important mindset in TKS of “figure it out” although, there were times where that mindset brought no comfort at all so in such moments sharing out her feelings with friends got her through it.
This is more of a personal lesson for you who’s reading this now — there will be times where all those inspirational quotes or mindsets only make you feel worse rather than encouraged in such times embrace the weakness and vulnerability of our human nature but don’t let it engulf you, at the end of the day what doesn’t kill you makes you stronger.
At some point Steph thought that she could perhaps move her AI to a local environment as the tutorial had a guidance for that approach however things didn’t workout and everyday was a painful one. She finally had to accept the fact that she wasn’t capable of such skill YET and rather than feeling like a failure she simply became more aware of her current skillset and ability and played to her strengths by focusing back on using GoogleColab for her AI. This is another important lesson to take note of but I’ll let you figure it out yourself, let's get back to the story.
Through every woe, peril and challenge more and more pain got inflicted on Steph’s mental state and eventually it caught up with her physical state too, however those occasions where she demonstrated not only resilience but also patience to confront the issue with a cool head is what truly got her to succeed in the end. Patience is key.
4: The conclusion
Just to make this clear — this is not yet the end of Steph’s journey. Rather one chapter closer to the end goal of having a functional app for this AI. As I’m writing this now its 1.25am and I’m waiting for my AI to finish training through 100 epochs, if not the GoogleColab will disconnect and all progress will be lost.
So that’s the end of the story for now, hope you enjoyed it as much as I did! Below are the links to the resources I made use of to improve my AI by switching to a YOLO structure, feel free to check it out if interested:
- Yolov8 tutorial
- Github computer vision guide
- Labelling annotations on images
- My discussion with perplexity AI
Until next time, and as always Remember Knowledge is Power.
