How To Create Your Own Seo Tool

01 Dec

The CTO of RankActive Reveals How to Build Your Own SEO Software

The web is flooded with tools promising to facilitate the work of SEO experts and digital marketers. You might have heard of Google Analytics, SEMrush, Moz, RankActive, and a myriad of other tools that claim to help to cope with paid search, SMM, email marketing and SEO. Some of these solutions are cost-free while others charge huge amounts of money.

Nevertheless, sometimes you don't get what you were looking for, or you just don't feel like paying for it. That is when you'd start thinking about the development of your SEO software.

But how do we know where to start and what to leave out?

We turned to a true expert on the subject: Evgeniy Vorobyov, the Chief Technical Officer at RankActive, the leading SEO platform with over 400 active users.

We asked Evgeniy about the process of SEO software development, which, surprisingly, appeared to be more complex than it may seem at first glance.

Evgeny Vorobyov

Tell us a bit about your background. What prompted you to develop your SEO software?

We started our business in 2009 when the SEO industry was still in its infancy. It was right before the launch of stricter regulations on the quality of content and keywords alongside the over-optimization. Back then, everyone seemed to be following black-hat practices. However, we predicted significant changes in Google Algorithm and established Areanda, one of the first white-hat SEO agencies in this part of the world. By refusing to jump on the bandwagon with big trends, we took off and assured the success of our small startup.

I came up with an idea of rank tracking software after years of using third-party solutions. I realized that we would need more than the market could give us for an acceptable price. We had to make a shift from inefficient and expensive tools and develop our own rank tracking software, the one that would fit our business model. At the same time, we have had to fulfill the needs of a customizable and flexible software for extracting SERP data, analyzing competitors' traffic, and obtaining keywords data on the top of Google Adwords.

So, we built an internal SEO toolkit, which later became a popular product. Today we have more than 400 active users of RankActive, and this number is increasing rapidly.

What role does the API play in the development of SEO tools?

Well, API plays a significant role here, it's basically the core of any software. The idea is to have your software talk directly with Google. However, Google doesn't sell its API, so you need an external "leverage" to extract the data from SERP and analyze it.

You can use different APIs for different purposes. For example, there is Google's analytics.js that allows tracking user-site interactions, Bing's Web Search API for retrieving information from SERP, and so on and so forth.

Nevertheless, a lot of API providers are making huge profits by setting high prices. They'd charge a lot of money for every API call you make and highly restrict the number of calls. The processing speed also leaves much to be desired.

We've chosen DataForSEO because their solutions are the most appropriate for the needs of our business. Having the most affordable APIs, they allow requesting the results of a particular task a few times. We can send up to 2000 API calls in a minute, which is very substantial. What's more, their processing speed is far better compared to other API providers, because at DataForSEO they don't use proxy websites in the classic sense. Instead, there are several data sources that help to diversify the reception of data and return location-specific results.

Let me get this straight. APIs that use conventional proxies return results in the raw HTML, along with captchas and all the unnecessary data. The more you use such API, the more "trash" you get. Basically, such solutions are like a huge money pit: you pay money for gigabytes of unnecessary data, not for needed results. By contrast, DataForSEO charges you for results only, so they guarantee that the money is invested in useful data.

Another thing to think about is the sort of data that you need for your project. For example, some are happy with rankings, while others are looking for keyword data like daily impressions, CPC, search volume, etc. DataForSEO provides well-rounded solutions for different purposes.

The last but not least is that we didn't have to create any projects, campaigns and make all these extra steps to integrate with DataForSEO. Their APIs can be integrated with a few simple steps, which allows us to focus on the development of our system.

How long did it take to get RankActive off the ground?

Well, it took us about a year of hard work to develop the core of our first product – Rank Tracker. We'd then spend several months working on the UI and UX.

Why did it take us so long? Well, first things first, software development is not a place to cut corners. You have to make sure that everything works smoothly and the system is stable beforehand. Moreover, we had to develop our API, which was probably the hardest part of a plan.

Day-to-day storing of TOP30 rankings for 1,000 keywords in Google alone generates unbelievable volumes of data in a relational database, which requires dozens of servers and hundreds of hard disks.

If we were to devise the same system today, it would be much easier. You have dozens of ready-use APIs and all these computing frameworks (such as Apache Spark, for example). It helps to cut the chase and substantially reduce the development time. I'd say that efficient API and Spark can minimize the entire process down to the 2-3 months period.

At the same time, it's only the case if you are building a rank tracking software. If you want to add SERP results, keywords data, competitor analysis, etc., the development time is likely to increase exponentially. I think that two-three years of dedicated work would be more realistic.

Is UI/UX important for SEO software?

The user interface may not seem that crucial when your software is only for internal use. You can always explain to your team members how to interact with the platform and get needed results.

Nevertheless, having a bigger picture always pays off. What if one day you'll decide to push your software on the market?

In this case, making a good-working tool is not enough. It should also be good-looking and easy to work with. The thing that a lot of software developers do not understand is that users don't want to know how great the architecture is. They want a product that would make their lives easier. If users don't get how to interact with your tool, they are not going to read your help center or watch video tutorials. They'll just start looking for another solution to the problem.

How did you find the right people for developing and running the software?

When we decided to kick off the development of our SEO tools, there were already a couple of talented programmers. Back then, it was enough for us to build up the architecture and deliver it as our in-house software.

However, if we're talking about a tool that would be competitive on the market, there is usually a lot of people involved. At different stages of the software development process, you may need some of the following skills (or all of them together): Chief Technical Architect, Application Architecture Specialist, UI and UX designers, system administrators, project manager, DBAs and lots of software developers.

How did I found all these people? Well, you better ask those questions to our HR managers 🙂

What challenges did you face at the early development stages?

Well, every development stage, every little detail is a challenge. The main issue, though, was devising the right architecture. You need an adequate scaling system to adapt the software to the growing user base.

Let me explain this a little bit. Let's say, you are trying to build a sort of keyword position tracking tool. You have this number of API calls divided among that number of users. And then your user database expands significantly. If you have poor architecture at the heart of your software, the system will collapse. That's the issue we were troubled about the most at the early development stages of RankActive.

Another thing to think about is customer support. The high-quality support team helped us to stand out among the competitors, but again, it was challenging to find the right people to get the work done.

Read more about the customer support team behind DataForSEO >>

How did you survive in the highly competitive environment? There are so many SEO software providers, what helps you to keep pace?

Yes, the market is competitive. New SEO software solutions are getting launched every month, and just as many of them drop the ball.

What helps us to keep pace? Well, we developed a narrower niche and focused on a few core fields where we have a competitive advantage. In the case of RankActive, there is everything you need to run a successful SEO campaign, so comprehensiveness of the information we provide is a key benefit that helps us to stay ahead of the curve.

Are there any marketing strategies that you would recommend to our readers?

We are making good use of both inbound and outbound marketing strategies. These two should be working all together.

However, outbound is getting less and less effective over the years. People are figuring out creative ways to block out our emails. What's more, it becomes costly.

So, I'd recommend to learn about inbound marketing and put more effort in things like SEO, social media and blogging.

And the last question. Can anyone build SEO software? What sort of advice would you give to those who decided to go down that road?

Absolutely. Talented software developers tend to be made, not born. The question is how much time, money and effort you are ready to invest in order to build a solid product. On the other hand, it's often a long shot. You'd spend thousands of hours struggling first with the development of your first tool, and then just as many hours would go to getting the bugs out.

If you can't decide whether to kick off your SEO software or buy a ready-made solution, check our blog post "Buy vs Build SEO Software: How to decide?"

As for my pieces of advice to newbies, well, first of all, I'd recommend focussing on a narrow field, something that is often neglected by the major market holders. That can be tools for finding long-tail conversational keywords or voice search optimization.