How to optimize your site for better findability

6 How to optimize your site for better findability

“You wouldn’t build a house without having a strong foundation,” Niki Mosier, head of SEO and content at AgentSync said, “And you shouldn’t build a website without having a strong foundation either, and without constantly making sure that foundation is strong and that there are no cracks in it.”

Optimizing the architecture of your site can help search engine crawlers find and index your content, which enables them to show those pages to users in the search results. It can also help send link authority around your site as well as make it easy for visitors to find what they’re looking for.

In her session at SMX Create, Niki Mosier shared the strategies she uses to ensure that the foundations of her site are solid and identify opportunities for greater search visibility.

Crawl budget analysis

Crawl budget refers to the number of URLs per site that Googlebot (or any other search engine crawler) can and wants to crawl.

“Every website gets a crawl budget, which can vary depending on the size of the site and the frequency that new content is being published on the site, so having an idea of what a website’s crawl budget is can be really beneficial in making informed decisions on what to optimize,” Mosier said.

Conducting a crawl budget analysis enables you to get a more comprehensive view of:

  • How your website is being crawled. “If you identify that Googlebot is the client, you can use log file analysis to find out how Googlebot is handling the URLs on your site [and] if it is crawling any pages with parameters,” she said.
  • How fast your site is. While there are many tools that can tell you how fast your server reacts, a log file analysis shows you how long it’s taking for a bot to download a resource from your server.
  • Indexing problems. “Getting into the log files can really show us whether bots are having trouble downloading a page fully,” Mosier said.
  • How often a URL is being crawled. The crawl frequency can be used to figure out if there are URLs that a search engine crawler should be crawling but isn’t, or vice versa.
  • Crawling problems. This tactic can also reveal when a crawler is encountering 404 errors or redirect chains, for example.

“When it comes to actually doing crawl budget analysis, there’s a couple of tools that are helpful,” Mosier said, recommending ScaremingFrog’s Log File Analyser, Microsoft Excel and Splunk.

Image: Niki Mosier.

Mosier outlined her steps to performing a crawl budget analysis:

  1. Obtain your log files; Mosier recommended working with at least a month of data.
  2. Look at URLs with errors.
  3. Assess which bots are crawling which areas of your site.
  4. Evaluate by day, week and month to establish patterns that may be useful for analysis.
  5. See if a crawler is crawling URLs with parameters, which may indicate wasted crawl budget.
  6. Cross-reference crawl data with sitemap data to assess for missed content.
SMX-Create-Niki-Mosier_s30-800x450-1 How to optimize your site for better findability
Image: Niki Mosier.

“Once you’ve dived into the server logs and have a good sense for what your crawl budget looks like, you can use this data to prioritize your SEO tasks,” she said, adding that SEOs should “prioritize based on the impact that fixing different areas of your site will have, the dev resources needed to fix issues and the time to fix those issues.”

RELATED: How to optimize your website’s crawl budget

Driving traffic with technical SEO

Finding out how well your site is functioning can help you put the right strategies in place to drive more traffic to it.

SMX-Create-Niki-Mosier_s40-800x450-1 How to optimize your site for better findability
Image: Niki Mosier.

“Doing regular site audits is a great way to keep a pulse on what’s happening with our websites,” Mosier recommended. In addition, Google Search Console should be used to check for Core Web Vitals or schema issues, for example. “Using monitoring tools, [such as] Rank Ranger, Semrush and Ahrefs, these are great ways to stay alerted to any issues that might pop up with your website,” she said.

Assessing the search results pages (SERP) can give you a feel for the landscape of the keywords you’re targeting. In addition to seeing what search features may be available, the SERP also shows you which sites are ranking higher than you — “See what those sites are doing; looking at their source code can tell you what schema they’re using,” Mosier said, adding that you should also be viewing their pages to scope out what their headings and user experience look like.

Updating your old content can also result in a rankings boost. Mosier recommends paying extra attention to your headings and above-the-fold content. Adding schema markup may also enable your content to appear as a rich result, which may also increase your visibility on the SERP.

“Using tools like Frase or Content Harmony can help you see what other sites that are ranking for the keywords that you want to be ranking for are using for headings, what kind of FAQ content they’re using and what content they have above the fold,” she added.

“Paying attention to page speed is definitely an important metric to think about, [but] I think it’s also important to pay attention to what the industry average is,” Mosier said, “So, go and look at where your competitors’ sites are ranking or are at as far as page speed and kind of set that as your benchmark.”

It’s also important to assess individual page speed versus overall site speed: “You want to see what each page on your site is loading for and make improvements on a page-by-page basis and not just look at the site speed as a whole because pages are what is ranking, not necessarily the whole site,” she said.

Additionally, how your pages render can affect your user experience as well as what search engine crawlers “see.” “Is there a pop-up or a really big header on a particular page that’s taking up a lot of the above-the-fold space? That can be a problem,” Mosier said, noting that page speed can also impact how search engines render a page.

More from SMX

New on Search Engine Land

About The Author

GeorgeNguyen-lg-20 How to optimize your site for better findability

George Nguyen is an editor for Search Engine Land, covering organic search, podcasting and e-commerce. His background is in journalism and content marketing. Prior to entering the industry, he worked as a radio personality, writer, podcast host and public school teacher.


Find Out More

Marketing Tips You Need

Keep In Touch

Quick Subscribe

Client Reviews Tell The Tale.

Nicole NoblesApril 18, 2024
Dan was a delight to work with. I needed a few headshots taken for my LinkedIn profile and Dan provided the easiest and most comfortable experience using state-of-the art equipment in a very professional setting. Also, the turn-around time on results was quick and I felt completely engaged and satisfied during the entire process. I highly recommend his services.Donny RitcharoenDecember 19, 2023
I got headshots taken and they turned out so well! The lighting was amazing.Tessa ChanMay 31, 2023
We used Appture to build a lodging website, and they were awesome! Dan went above and beyond to show us the functions and make all of our changes. Appture is our go to for web design from now on!Abigail HaleOctober 26, 2022
Dan did a fantastic job making me feel comfortable while shooting. He also made me look great! I don't photograph well, so I am very pleased with the results and speed at which I got the final product.Lily GostinSeptember 13, 2022
Appture knows their business and will go the extra mile for their customers. They do high quality work and provide great ongoing support.Chris McCorkindaleMay 25, 2022
Anita CauthornMay 24, 2022
It’s so rare in these times to find one man with so much wow factor and more rare to find men with similar interest and passion in their life journey as myself . Dan Elliott has been introduced to many in what is now considered as the Terror Dome , a place where many dreams are not deferred they are detoured to routes that lead to dead ends , he comes in full of optimism so infectious that he, maybe with out knowing is energizing those who have ventured where others would fear going with just the right jolt to forge on in the way of helping fallen humanity … His various fields of expertise has helped many in my region and I can only imagine the number he has effected beyond those I know … from day one I knew “ this was a man of kindred spirit “ Dan Elliott is a Gem and adds glimmer to things he touches … I’m a Witness ….and eternally grateful….L.Rashaan RichMay 21, 2022
Dan and his group are highly capable and knowledgeable. They work fast and get the job done. I highly recommend Appture.Justin FrankMarch 26, 2022
They are highly specialized in their work and constantly seek innovation.Ismail YenigulMarch 14, 2022
Dan is a marketing wizard. Honest, Experienced and a read deal. I am blessed to have him in my journey online :) Highly recommended.Sabbir HasanMarch 7, 2022
So much to say. Creative, Intelligent, Talented, Limitless, Affordable. It's amazing what these guys can do.Hack mackMay 17, 2019
We'd used some other agencies before, but man, they simply knocked us all over. After being in business for 30 years, I wonder how much more business we'd be doing if we'd hired them earlier.Rebecca HoneaMay 17, 2019