Show me the money!!!
Are you ready to raise money for your startup?
Leslie Mitts, Managing Practice Leader at the Wharton SBDC and Lead
Advisor for the Wharton Venture Initiation Program, tells us that most
entrepreneurs coming through her programs are focused on raising
capital, even though there are higher priorities in many of their
businesses. Since fresh dollars help drive the business engine, this is
a natural entrepreneurial concern.
But are you ready to raise money?
Going to investors before you are really ready can damage your reputation and may actually hinder future funding efforts.
Team and a Dream uses a unique Fundability Diagnostic to help companies
assess their strengths in four key areas:
Team - This is the most frequently cited attribute examined by
investors. Many investors will invest in an A-level team and a B-level
product, but not the opposite. Having your Seed Team nailed is crucial.
Traction - Showing momentum is a critical element to demonstrating
progress and your ability to execute.
Perceived Pain - Do people really need your product? Why? Will they
need it in the future?
Product Eco-system - How does your product fit into existing and
emerging landscapes? Who else is out there and how big will this market
grow?
Find out what you need to succeed and discover your roadmap to getting
there. Take the quiz below to see how well you might expect to score
with investors. For each of the four sections outlined above, see
which hurdles you can clear given the current state of your business.
PROCEED WITH CAUTION: Only self-aware entrepreneurs should take these
results seriously.
ASSESSING YOUR TEAM
Part A:
If you or any member of your current team are not planning on being the
CEO, give yourself two points and proceed to Part B.
If someone on your team IS planning to continue on as CEO, then youÂ’ll
need to answer yes to both of the next two questions in order to score
two points; otherwise, take a zero.
1. Our CEO has significant, previous management experience in a startup
or comparable operating environment (i.e. middle management in a large
company doesnÂ’t count!) ( 1 point)
2. Our CEO is able to rapidly assess and prioritize our work with the
respect of the team ( 1 point)
Part B:
Score one point for each answer.
1. Someone on my team has significant experience in the domain space in
which we are starting the company ( 1 point)
2. Our team understands and acts as if they understand the difference
between tasks that are urgent, urgent and important, important, and
none of the above ( 1 point)
3. This team has terrific chemistry ( 1 point)
Your score (0-5) is your Total Team Score.
If you scored three or more, you pass the Team Hurdle.
TRACTION
Score one point for each item you can answer positively.
1. We have made a few initial sales and have positive indicators from
the marketplace for additional sales growth ( 1 point)
2. So many good things are happening I am bouncing off the walls with
excitement ( 1 point)
3. I have added a significant team member (besides myself!) in the past
two months ( 1 point)
4. In the past two months, I received press coverage that would be
acknowledged by my prospects as legitimate business coverage ( 1 point)
5. We recently have achieved a significant product or marketplace
milestone (includes a single, initial sale) ( 1 point)
Your score (0-5) is your Total Traction Score.
If you scored three or more, you pass the Traction Hurdle.
PERCEIVED PAIN
Award your Team the number of points indicated after each question that
you answer positively.
1. My product or service solves an important problem for a well defined
group of people ( 2 points)
2. My product or service will be in great demand in the future in order
to solve problems emerging from a new trend that has a large population
of prospects ( 2 points)
3. .My product or service takes advantage of newly available market
forces (technology, demographics, societal). The result is an offering
that will clearly appeal to an existing, well-known market demographic
( 1 point)
Your score (0-5) is your Total Perceived Pain Score.
If you scored three or more, you pass the Perceived Pain Hurdle.
PRODUCT ECO-SYSTEM
Score the stated number of points for each item you can answer
positively.
1. My product is unique in a clearly defined (i.e. I can describe the
difference in one or two clear sentences) and useful way ( 1 point)
2. My product is patented or can easily be patented or I have another
competitive barrier (e.g. trade secret, unique manufacturing process,
etc.) that will make it difficult for others to compete directly ( 1
point)
3. My marketplace is well established but fragmented or my product has
“category killer” potential because it is so unique to the market ( 1
point)
4. My marketplace is either very big (> $5 billion/year in sales) with
clear room for a product like mine or is rapidly growing with well
documented trend lines ( 1 point)
5. My marketplace is wide open and there are no competitors ( subtract 1
point)
Your score of (-1 to 4) is your Total Product Eco-System Score.
If you scored three or more, you pass the Product Eco-System Hurdle.
Now that you have completed the scoring, we shall take a look at how
“fundable” your company is. Before we do, it’s important to realize
that this is a snapshot in time. If you are not satisfied with your
score, there are plenty of things you can do to improve it. IÂ’ve seen
many of my clients make tremendous progress along these dimensions by
extending their team, focusing on sales or tweaking their offering.
YOUR FUNDABILITY RATING:
Number of Hurdles Cleared: None
IMPLICATION
If you received this score and hope to receive funding, then you have a
lot of work to do.
Perhaps you just know you want to start a business and are looking for
ideas. If so, keep these criteria in mind as you formulate your plans.
If you already have a business concept, you should think long and hard
about “re-conceiving” some elements of it to improve your scores.
Many perfectly good “lifestyle” businesses may fall into this scale –
you just canÂ’t get funding for them.
Number of Hurdles Cleared: One
IMPLICATION
Funding from your rich uncle may be possible
Number of Hurdles Cleared: two
IMPLICATION
Funding from family and friends network may be possible
Number of Hurdles Cleared: three
IMPLICATION
Professional investors will consider this, but youÂ’ll need to be
prepared to explain how youÂ’ll address your gaps.
Number of Hurdles Cleared: all four
IMPLICATION
Put the finishing touches on your business plan and hit the funding
trail!
You will have the best chances of success in fund-raising if investors
see that you can clear the hurdles in all four areas.
NOTE: Team and a Dream offers a professional version of this assessment
that offers additional detail and examines nuances that arenÂ’t possible
to convey in a simple quiz. If you have an interest in the
professional assessment, please visit us at
http://www.teamandadream.com/contactus.htm
Contact Information:
http://www.teamandadream.com
skip@teamandadream.com
(610) 687-9195
Copyright (c) 2005, All rights reserved by Post DestinyBusiness Management Articles, Inc. d/b/a
Team and a Dream(SM)
ABOUT THE AUTHOR
As an advisor-builder with a sense of adventure, founder Skip Shuda has worked with numerous startups since 1983, including Destiny WebSolutions, which he founded in 1994. Destiny was named Startup Company of the year in 1998's Philadelphia Area Enterprise Awards. Destiny grew to $25 million in annualized revenues in 2000 and employed over 120 people.
Sunday, May 13, 2007
Starting An Online Business Can Be Fun And Profitable
Starting an online business can be fun, profitable and a very rewarding experience. Many people dream of having a home based business and the freedom that comes with a successful venture. There are many online business opportunities to choose from. Just go to any search engine and type in “home business opportunities” and you will literally have thousands of opportunities. So how do we decide which one is right for you? The right opportunity depends on several factors. Two key factors are how much time you can devote to your new business venture and how much your budget can afford.
The best online business opportunities should be free to join, have multiple affiliate programs to choose from, and offer free website design and setup. This type of opportunity is totally free and provides residual income streams. Your only monetary requirement is to pay for website hosting, an autoresponder service, and advertising your new website. Your website hosting and autoresponder service can vary significantly, so check pricing before committing, and your advertising budget is totally up to you. If your advertising budget is tight, there are several free forms of advertising available. Take things slow and don’t get discouraged. It takes time to build a business.
Your new online business should first begin as a “sole proprietorship”, which means that you are the sole owner of the business. Your social security number will be your tax identification number, so you don’t need to apply for a federal identification number. You don’t even have to report to the government that you are in business until you file your tax return (schedule C).
Remember that every expense for your business is tax deductible. Your new business can generate thousands of dollars in deductible expenses every year whether your business is profitable or not. Keep receipts and a record of your income and expenses. This can be easily accomplished with any spreadsheet software and is probably already on your computer.
Once your website is up and running and your autoresponder is ready to send prewritten articles to all your potential customers, your next step is to begin your advertising campaign. You have several advertising options to choose from. If you desire to get off to a quick start then you will have to pay for advertising. This can be in the form of purchasing leads from a reputable lead generating company, buying key word phrases from places like Google or Overture, ad placement on various websitesFind Article, and traffic generating sites. Success in your new online business venture depends on your marketing effort. Online business success is 90% marketing and only 10% dependent on the products or services you offer. The key to your success is marketing!
One of the best advertising options for tight budgets is free traffic generating sites. Go to any search engine and type in “free web traffic” and visit several of the listings. The best sites will enable you to generate free traffic at no cost. These “free sites” allow you to surf for credits. These credits are then used to promote your website. Another free option is linking your website to another website. This is called reciprocal linking and can generate a steady flow of potential customers to your website.
ABOUT THE AUTHOR
David Clark operates a successful online business for fun and for profit. Launch your very own money making website today that's 100% ready to take orders and pull in MASSIVE PROFITS for you right now...GUARANTEED! Don't delay visit: http://www.moneybuildersystem.com/pips.html
The best online business opportunities should be free to join, have multiple affiliate programs to choose from, and offer free website design and setup. This type of opportunity is totally free and provides residual income streams. Your only monetary requirement is to pay for website hosting, an autoresponder service, and advertising your new website. Your website hosting and autoresponder service can vary significantly, so check pricing before committing, and your advertising budget is totally up to you. If your advertising budget is tight, there are several free forms of advertising available. Take things slow and don’t get discouraged. It takes time to build a business.
Your new online business should first begin as a “sole proprietorship”, which means that you are the sole owner of the business. Your social security number will be your tax identification number, so you don’t need to apply for a federal identification number. You don’t even have to report to the government that you are in business until you file your tax return (schedule C).
Remember that every expense for your business is tax deductible. Your new business can generate thousands of dollars in deductible expenses every year whether your business is profitable or not. Keep receipts and a record of your income and expenses. This can be easily accomplished with any spreadsheet software and is probably already on your computer.
Once your website is up and running and your autoresponder is ready to send prewritten articles to all your potential customers, your next step is to begin your advertising campaign. You have several advertising options to choose from. If you desire to get off to a quick start then you will have to pay for advertising. This can be in the form of purchasing leads from a reputable lead generating company, buying key word phrases from places like Google or Overture, ad placement on various websitesFind Article, and traffic generating sites. Success in your new online business venture depends on your marketing effort. Online business success is 90% marketing and only 10% dependent on the products or services you offer. The key to your success is marketing!
One of the best advertising options for tight budgets is free traffic generating sites. Go to any search engine and type in “free web traffic” and visit several of the listings. The best sites will enable you to generate free traffic at no cost. These “free sites” allow you to surf for credits. These credits are then used to promote your website. Another free option is linking your website to another website. This is called reciprocal linking and can generate a steady flow of potential customers to your website.
ABOUT THE AUTHOR
David Clark operates a successful online business for fun and for profit. Launch your very own money making website today that's 100% ready to take orders and pull in MASSIVE PROFITS for you right now...GUARANTEED! Don't delay visit: http://www.moneybuildersystem.com/pips.html
Search Engines vs. SEO Spam: Statistical Methods
It takes months for the strategic SEO campaign to produce feasible effect, and in order to achieve fast results many webmasters and Internet marketers use so-called 'black hat' SEO techniques. Search engines continiously battle web spam produced by 'black-hat' SEO - millions of pages created to manipulate search results by spammy linkage, keyword stuffing and other tricks. In my article I describe several statistical methods used by search engine to detect such pages.
Originally published at: Search Engines vs. SEO Spam: Statistical Methods
High placement in a search engine is critical for the success of any online business. Pages appearing higher in the search engine results to queries relevant to a site's business will get higher targeted traffic. To get this kind of competitive advantage Internet companies employ various SEO techniques in order to optimize certain factors used by search engines to rank results. In the best case SEO specialists create relevant well-structured keyword rich pages, which not only please the eyes of a search engine crawler but also have value to the human visitor. Unfortunately it takes months for this strategic approach to produce feasible results, and many search engine optimizers use so-called "black-hat" SEO.
'Black Hat' SEO and Search Engine Spam
The oldest and simplest "black SEO" strategy is adding a variety of popular keywords into web pages to make them rank high for popular queries. This behavior is easily detected since generally such pages include unrelated keywords that lack topical focus. With the introduction of the term vector analysis search engine became immune to this sort of manipulation. However "black-hat' SEO went one step further creating the so-called "doorway' pages - tightly focused pages consisting of a bunch of keywords relevant to a single topic. In terms of keyword density such pages are able to rank high in search results but never seen by human visitors as they are redirected to the page intended to receive the traffic. Another trend is the abusing the link popularity based ranking algorithms, such as PageRank with the help of dynamically-generated pages. Such pages receive the minimum guaranteed PageRank and the small endorsements from thousands of these pages are able to produce a sizeable PageRank for the target page. Search engines constantly improve their algorithms trying to minimize the effect of "black-hat"' SEO techniques, but SEOs also persistently respond with new more sophisticated and technically advanced tricks so that this process bears a resemblance to an arms race.
"Black-hat" SEO is responsible for the immense amount of search engine spam -- pages and links created solely to mislead search engines and boost rankings for client web sites. To weed out the web spam search engines can use statistical methods that allow computing distributions for a variety of page properties. The outlier values in these distributions can be associated with web spam. The ability to identify web spam is extremely valuable to search engine not just because it allows excluding spam pages from their indices but also using them to train more sophisticated machine learning algorithms capable to battle web spam with higher precision.
Using Statistics to Detect Search Engine Spam
An example of an application of statistical methods to detect web spam is presented in the paper "Spam, Damn Spam and Statistics" by Dennis Fetterly, Mark Manasse and Marc Najork from Microsoft. They used two sets of pages downloaded from the Internet. The first set was crawled repeatedly from November 2002 to February 2003 and consisted from 150 million URLs. For each page the researches recorded HTTP status, time of download, document length, number of non-markup words, and a vector indicating the changes in page content between downloads. A sample of this set (751 pages) was inspected manually and 61 spam pages were discovered, or 8.1% of the set with a confidence interval of 1.95% at 95% confidence.
Another set was crawled between July and September 2002 and comprises 429 million pages and 38 million HTTP redirects. For this set the following properties were recorded: URL, URLs of outgoing links; for the HTTP redirects - the source and the target URL. 535 pages were manually inspected and 37 of them were identified as spam (6.9%).
The research concentrates on studying the following properties of web pages:
* URL properties, including length and percentage of non-alphabetical characters (dashes, digits, dots etc.).
* Host name resolutions.
* Linkage properties.
* Content properties.
* Content evolution properties.
* Clustering properties.
URL Properties
Search engine optimizers often use numerous automatically generated pages to massively distribute their low PageRank to a single target page. Since the pages are machine generated we can expect their URLs to look differently from those created by humans. The assumptions are that these URLs are longer and include more non-alphabetical characters such as dashes, slashes or digits. When searching for spam pages we should consider the host component only, not the entire URL down to the page name.
The manual inspection of the 100 longest hostnames had revealed that 80 of them belong to adult site and 11 refer to the financial and credit related sites. Therefore in order to produce a spam identification rule the length property has to be combined with the percentage of non-alphabetical characters. In the given set 0.173% of URLs are at least 45 characters long and contain at least 6 dots, 5 dashes or 10 digits -- and the vast majority of these pages appear to be spam. By changing the threshold values we can change the number of pages flagged as spam and the number of false positives.
Host Name Resolutions
One can notice that Google, given a query q, tends to rank a page higher if the host component of the page's URL contains keywords from q. To utilize this search engine optimizers stuff pages with URLs containing popular keywords and keyphrases and set up DNS servers to resolve these URLs to a single IP. Generally SEOs generate a large number of host names to rank for a wide variety of popular queries.
This behavior can also be relatively easy detected by observing the number of host name resolutions to a single IP. In our set 1,864,807 IP addresses are mapped to only one host name, and 599,632 IPs -- to 2 host names. There are also some extreme cases with hundreds of thousands host names mapped to a single IP, and the record-breaking IP referred by 8,967,154 host names.
To flag pages as spam a threshold of 10,000 name resolutions was chosen. About 3.46% of the pages in the Set 2 are served from IP addresses referred by 10,000 and more host names and the manual inspection of this sample proved that with very few exceptions they were spam. Lower threshold (1,000 name resolutions or 7.08% pages in the set) produces an unacceptable amount of false positives.
Linkage Properties
The Web consisting of interlinked pages has a structure of a graph. Therefore in graph terminology the number of outgoing links of a page can be referred to as the out-degree, while the in-degree equals to the number link pointing to a page. By analyzing out- and in-degrees values it is also possible to detect spam pages which would represent the outliers in the corresponding distributions.
In our set for example there are 158,290 pages with out-degree 1301, while according to the overall trend only 1,700 such pages are expected. Overall 0.05% of pages in the Set 2 have out-degrees at least three times more than suggested by the Zipfian distribution, and according to the manual inspection of a cross section, almost all of them are spam.
Similarly the distribution for in-degrees is calculated. For example 369,457 pages have the in-degree of 1001, while according to the trend only 2,000 such pages are expected. Overall, 0.19% of pages in the Set 2 have in-degrees at least three times more common than the Zipfian distribution would suggest, and the majority of them are spam.
Content Properties
Despite the recent measures taken by search engines to diminish the effect of keyword stuffing, this technique is still used by some SEOs who generate pages filled with meaningless keywords to promote their AdSense pages. Quite often such pages are based on a single template and even have the same number of words which makes them especially easy to detect using statistical methods.
For Set 1 the number of non-markup words in each page was recorded, so we can draw the variance of word count in pages downloaded from a given host name. The variance is plotted on the x-axis and the word count is shown on the y-axis, both axes are drawn on a logarithmic scale. Points in the left side of the graph marked with blue represent cases where at list 10 pages from a given host have the same word count. There are 944 such hosts (0.21% of the pages in Set 1). A random sample of 200 these pages was examined manually: 35% were spam, 3.5% contained no text and 41.5% were soft errors (a page with a message indicating that the resource is not currently available, despite the HTTP status code 200 OK ).
Content Evolution
The natural evolution of the content in the Web is slow. In a period of a week 65% of all pages will not change at all, while only 0.8% will change completely. In contrast many spam SEO web pages generated in response to an HTTP request independent of the requested URL will change completely of every download. Therefore by looking into extreme cases of content mutation we search engines are able to detect web spam.
The outliers represent IPs serving the pages that change completely every week. Set 1 contains 367 such servers with 1,409,353 pages (97.2%). The manual examination of a sample of 106 pages showed that 103 (97.2%) were spam, 2 were soft errors and 1 adult pages counted as a false positive.
Clustering Properties
Automatically generated spam pages tend to look very similar. In fact, as already said above, most of them are based on the same model and have only minor differences (like inserting varying keywords into a template). Pages with such properties can be detected by applying clustering analysis to our samples.
To form clusters of similar pages the 'shingling' algorithm described by Broder et al. [2] will be used. Figure 7 shows the distribution of the cluster sizes on near duplicate pages in Set 1. The horizontal axis shows the size of the cluster (the number of pages in the near-equivalence class), and the vertical axis shows how many such clusters Set 1 contains.
The outliers can be put into two groups. The first group did not contain any spam pages, pages in this group are more related to the duplicated content issue. In the same time the second group is populated predominantly by spam documents. 15 of 20 largest clusters were spam containing 2,080,112 pages (1.38% of all pages in Set 1)
To Sum Up
The methods described above are the examples of a fairly simple statistical approach to spam detection. The real life algorithms are much more sophisticated and are based on machine learning technologies which allow search engine to detect and battle spam with a relatively high efficiency at an acceptable rate of false positives. Applying the spam detection techniques enables search engine to produce more relevant results and ensures a more fair competition based on the quality of web resources and not on technical tricks.
References:
1. Dennis Fetterly, Mark Manasse, Marc Najork. "Spam, Damn Spam, and Statistics: Using statistical analysis to locate spam web pages" (2004). Microsoft Research. Available at: http://research.microsoft.com/~najork/webdb2004.pdf
2. A. Broder, S. Glassman, M. Manasse, and G. Zweig. "Syntactic Clustering of the Web". In 6th International World Wide Web ConferenceFeature Articles, April 1997.
Graphics omitted. The full version of the article can be found here: Search Engines vs. SEO Spam: Statistical Methods
ABOUT THE AUTHOR
Originally published at: Search Engines vs. SEO Spam: Statistical Methods
High placement in a search engine is critical for the success of any online business. Pages appearing higher in the search engine results to queries relevant to a site's business will get higher targeted traffic. To get this kind of competitive advantage Internet companies employ various SEO techniques in order to optimize certain factors used by search engines to rank results. In the best case SEO specialists create relevant well-structured keyword rich pages, which not only please the eyes of a search engine crawler but also have value to the human visitor. Unfortunately it takes months for this strategic approach to produce feasible results, and many search engine optimizers use so-called "black-hat" SEO.
'Black Hat' SEO and Search Engine Spam
The oldest and simplest "black SEO" strategy is adding a variety of popular keywords into web pages to make them rank high for popular queries. This behavior is easily detected since generally such pages include unrelated keywords that lack topical focus. With the introduction of the term vector analysis search engine became immune to this sort of manipulation. However "black-hat' SEO went one step further creating the so-called "doorway' pages - tightly focused pages consisting of a bunch of keywords relevant to a single topic. In terms of keyword density such pages are able to rank high in search results but never seen by human visitors as they are redirected to the page intended to receive the traffic. Another trend is the abusing the link popularity based ranking algorithms, such as PageRank with the help of dynamically-generated pages. Such pages receive the minimum guaranteed PageRank and the small endorsements from thousands of these pages are able to produce a sizeable PageRank for the target page. Search engines constantly improve their algorithms trying to minimize the effect of "black-hat"' SEO techniques, but SEOs also persistently respond with new more sophisticated and technically advanced tricks so that this process bears a resemblance to an arms race.
"Black-hat" SEO is responsible for the immense amount of search engine spam -- pages and links created solely to mislead search engines and boost rankings for client web sites. To weed out the web spam search engines can use statistical methods that allow computing distributions for a variety of page properties. The outlier values in these distributions can be associated with web spam. The ability to identify web spam is extremely valuable to search engine not just because it allows excluding spam pages from their indices but also using them to train more sophisticated machine learning algorithms capable to battle web spam with higher precision.
Using Statistics to Detect Search Engine Spam
An example of an application of statistical methods to detect web spam is presented in the paper "Spam, Damn Spam and Statistics" by Dennis Fetterly, Mark Manasse and Marc Najork from Microsoft. They used two sets of pages downloaded from the Internet. The first set was crawled repeatedly from November 2002 to February 2003 and consisted from 150 million URLs. For each page the researches recorded HTTP status, time of download, document length, number of non-markup words, and a vector indicating the changes in page content between downloads. A sample of this set (751 pages) was inspected manually and 61 spam pages were discovered, or 8.1% of the set with a confidence interval of 1.95% at 95% confidence.
Another set was crawled between July and September 2002 and comprises 429 million pages and 38 million HTTP redirects. For this set the following properties were recorded: URL, URLs of outgoing links; for the HTTP redirects - the source and the target URL. 535 pages were manually inspected and 37 of them were identified as spam (6.9%).
The research concentrates on studying the following properties of web pages:
* URL properties, including length and percentage of non-alphabetical characters (dashes, digits, dots etc.).
* Host name resolutions.
* Linkage properties.
* Content properties.
* Content evolution properties.
* Clustering properties.
URL Properties
Search engine optimizers often use numerous automatically generated pages to massively distribute their low PageRank to a single target page. Since the pages are machine generated we can expect their URLs to look differently from those created by humans. The assumptions are that these URLs are longer and include more non-alphabetical characters such as dashes, slashes or digits. When searching for spam pages we should consider the host component only, not the entire URL down to the page name.
The manual inspection of the 100 longest hostnames had revealed that 80 of them belong to adult site and 11 refer to the financial and credit related sites. Therefore in order to produce a spam identification rule the length property has to be combined with the percentage of non-alphabetical characters. In the given set 0.173% of URLs are at least 45 characters long and contain at least 6 dots, 5 dashes or 10 digits -- and the vast majority of these pages appear to be spam. By changing the threshold values we can change the number of pages flagged as spam and the number of false positives.
Host Name Resolutions
One can notice that Google, given a query q, tends to rank a page higher if the host component of the page's URL contains keywords from q. To utilize this search engine optimizers stuff pages with URLs containing popular keywords and keyphrases and set up DNS servers to resolve these URLs to a single IP. Generally SEOs generate a large number of host names to rank for a wide variety of popular queries.
This behavior can also be relatively easy detected by observing the number of host name resolutions to a single IP. In our set 1,864,807 IP addresses are mapped to only one host name, and 599,632 IPs -- to 2 host names. There are also some extreme cases with hundreds of thousands host names mapped to a single IP, and the record-breaking IP referred by 8,967,154 host names.
To flag pages as spam a threshold of 10,000 name resolutions was chosen. About 3.46% of the pages in the Set 2 are served from IP addresses referred by 10,000 and more host names and the manual inspection of this sample proved that with very few exceptions they were spam. Lower threshold (1,000 name resolutions or 7.08% pages in the set) produces an unacceptable amount of false positives.
Linkage Properties
The Web consisting of interlinked pages has a structure of a graph. Therefore in graph terminology the number of outgoing links of a page can be referred to as the out-degree, while the in-degree equals to the number link pointing to a page. By analyzing out- and in-degrees values it is also possible to detect spam pages which would represent the outliers in the corresponding distributions.
In our set for example there are 158,290 pages with out-degree 1301, while according to the overall trend only 1,700 such pages are expected. Overall 0.05% of pages in the Set 2 have out-degrees at least three times more than suggested by the Zipfian distribution, and according to the manual inspection of a cross section, almost all of them are spam.
Similarly the distribution for in-degrees is calculated. For example 369,457 pages have the in-degree of 1001, while according to the trend only 2,000 such pages are expected. Overall, 0.19% of pages in the Set 2 have in-degrees at least three times more common than the Zipfian distribution would suggest, and the majority of them are spam.
Content Properties
Despite the recent measures taken by search engines to diminish the effect of keyword stuffing, this technique is still used by some SEOs who generate pages filled with meaningless keywords to promote their AdSense pages. Quite often such pages are based on a single template and even have the same number of words which makes them especially easy to detect using statistical methods.
For Set 1 the number of non-markup words in each page was recorded, so we can draw the variance of word count in pages downloaded from a given host name. The variance is plotted on the x-axis and the word count is shown on the y-axis, both axes are drawn on a logarithmic scale. Points in the left side of the graph marked with blue represent cases where at list 10 pages from a given host have the same word count. There are 944 such hosts (0.21% of the pages in Set 1). A random sample of 200 these pages was examined manually: 35% were spam, 3.5% contained no text and 41.5% were soft errors (a page with a message indicating that the resource is not currently available, despite the HTTP status code 200 OK ).
Content Evolution
The natural evolution of the content in the Web is slow. In a period of a week 65% of all pages will not change at all, while only 0.8% will change completely. In contrast many spam SEO web pages generated in response to an HTTP request independent of the requested URL will change completely of every download. Therefore by looking into extreme cases of content mutation we search engines are able to detect web spam.
The outliers represent IPs serving the pages that change completely every week. Set 1 contains 367 such servers with 1,409,353 pages (97.2%). The manual examination of a sample of 106 pages showed that 103 (97.2%) were spam, 2 were soft errors and 1 adult pages counted as a false positive.
Clustering Properties
Automatically generated spam pages tend to look very similar. In fact, as already said above, most of them are based on the same model and have only minor differences (like inserting varying keywords into a template). Pages with such properties can be detected by applying clustering analysis to our samples.
To form clusters of similar pages the 'shingling' algorithm described by Broder et al. [2] will be used. Figure 7 shows the distribution of the cluster sizes on near duplicate pages in Set 1. The horizontal axis shows the size of the cluster (the number of pages in the near-equivalence class), and the vertical axis shows how many such clusters Set 1 contains.
The outliers can be put into two groups. The first group did not contain any spam pages, pages in this group are more related to the duplicated content issue. In the same time the second group is populated predominantly by spam documents. 15 of 20 largest clusters were spam containing 2,080,112 pages (1.38% of all pages in Set 1)
To Sum Up
The methods described above are the examples of a fairly simple statistical approach to spam detection. The real life algorithms are much more sophisticated and are based on machine learning technologies which allow search engine to detect and battle spam with a relatively high efficiency at an acceptable rate of false positives. Applying the spam detection techniques enables search engine to produce more relevant results and ensures a more fair competition based on the quality of web resources and not on technical tricks.
References:
1. Dennis Fetterly, Mark Manasse, Marc Najork. "Spam, Damn Spam, and Statistics: Using statistical analysis to locate spam web pages" (2004). Microsoft Research. Available at: http://research.microsoft.com/~najork/webdb2004.pdf
2. A. Broder, S. Glassman, M. Manasse, and G. Zweig. "Syntactic Clustering of the Web". In 6th International World Wide Web ConferenceFeature Articles, April 1997.
Graphics omitted. The full version of the article can be found here: Search Engines vs. SEO Spam: Statistical Methods
ABOUT THE AUTHOR
Oleg Ishenko, MCDBA, MCSE gives useful advice on search engine optimization at his blog SEO Research
Undocumented Aircraft Repairs
What is Undocumented Damage?
Undocumented damage is most likely the cause of an undocumented repair. The FARs do not discuss undocumented damage. They do however discuss undocumented repairs. These are repairs performed to and aircraft, engine propeller, or appliance that is not documented properly in the maintenance records. There are only a few reasons this would occur. We will discuss the reasons in this month's newsletter.
Unintentional Damage Examples
Hanger Rash
Hanger Rash is a common undocumented damage. Which can easily lead to an undocumented repair. Let’s say a lineman was re-positioning aircraft to get ready for the next day's departures when he accidentally clips a position light against the hanger door. He realizes this and sees that it pulled the light out from the wings fiberglass mounting surface just slightly. He, who is mechanically inclined but not authorized to do a repair, gets his tool box and removes the wingtip and institutes a substandard and illegal repair to prevent anyone from realizing that any damage occurred. Now he believes he has done the right thing, after all he has saved his company from the time and embarrassment associated with dealing with this problem and the customer can fly his plane the next morning. This is obviously a rare occurrence but none the less it happens all too often. This is an example where undocumented damage becomes an undocumented repair.
Bird Strike
Bird Strikes are even less common but do pose a real issue with undocumented damage. A pilot is on short final when a small flock of birds is startled and fly into his path. He receives a strike against his wing's leading edge and it dents slightly. This pilot who is an A&P looks at the damage and decides that it is negligible and does not report the damage to the owner. This bird strike was later seen by the owner and he asked the mechanic about it. The mechanic told him the story and said it was a negligible dent and wouldn't affect the flight characteristics of the aircraft. And besides "you don't want a damage history". Well this is entirely all too common. A mechanic advises his client to not document or repair negligible damage due to its effect on aircraft value. This is simply not good advice. At the end of the day the aircraft must conform to its original type design and any negligible damage is not part of the type design. I know what you’re thinking and the answer is "It is a very narrow line to walk but use your judgment on what should be repaired and what really is negligible." Regardless it must all be documented.
Hard Landings
This is common among training aircraft. The reality is that aircraft are very very tough. Sometime though, not tough enough. Some of the hidden damage could be overlooked for several inspection cycles and sometime maybe not discovered at all. It is important to make sure that when you experience a hard landing that you have the aircraft thoroughly inspected SPECIFICALLY for that damage. Do not wait until the next 100 hour and say "Oh yeah, I had a student really drop this one on its nose last month. Could you look for any damage?" What happens here is that the mechanic is performing a 100 hour inspection, not a hard landing inspection. The inspection should always equal the suspected damage. This is why when you suspect any damage you should inspect for that damage immediately. Don't put it off just because you don't see any obvious defects.
Un-Ethical Practices To Reduce Damage History
Believe it or not it happens. This is when a person provides substandard services, illegal repairs, when an owner "Fixes his plane", and when a person puts the future aircraft value ahead of safety and the rule of regulatory compliance. Where the events and scenarios above were fictitious these will be directly from my experience as a mechanic.
Cowling Alteration - Substandard Service
I was working at a facility where a light twin had been grounded for being un airworthy. The aircraft had some major issues but could be easily repaired. One of the damaged components was the aft mounting plates for the engine cowlings. The repair was completed and the mechanics moved on to the next issue. During this maintenance the aircraft received some hanger rash and a new elevator was required. This became the major issue and once resolved the aircraft was signed off as airworthy but the repair to the cowling was never documented. The service the owner received was not to the standards required by the FARs and the owner should have expected these mechanics to do a better job. Be very careful when hiring mechanics as they are not all equal.
Ice Shield Repair - Illegal Repairs
When I worked for an airline I was involved in a squawk that required a repair or replacement of a fuselage ice shield. The replacement part was in the computer inventory and I removed the damaged component. The question that was raised in this case is what happens if the replacement part is not actually in stock? Well unfortunately the broad answer is not so easy. My answer is you cannot reinstall that damaged item even if it was deferrable due to the fact that the steps in properly documenting a maintenance event is to record the removal of the damaged part. Once you have committed to removing the damaged part it becomes un-airworthy and cannot be re-installed regardless of its deferability in the MEL.
The actual events that occurred were as follows:
I noticed a significant impact to the ice shield and wrote it up as being damaged.
The inspector told me to replace the ice shield.
I confirmed the availability of the replacement part in the computer inventory.
I removed the damaged part and prepared the surface for the replacement part.
I sent for the new part and was told that we didn't have it in stock. (Inventory Error)
The Inspector said to re-install the old one.
I refused.
He said to repair it.
I researched the SRM and found that we did not have the proper facility to repair it.
He said to just put some 5 minute epoxy in the area and sand it down and paint it.
I walked away.
He found someone else to jeopardize their standards and the aircraft departed on time, illegal, and unairworthy.
This is a common practice and is overlooked by many people and can be considered as SOP for some maintenance facilities. It is the responsibility of the mechanic to stand firm on his knowledge of the regulations and question the actions of his supervisors if the policy is to circumvent the rules to make an on time departure.
Owner Repairs - Propeller Installation - Improper Authority
This is a case of blatant regulatory violation. A pilot who tied his aircraft down at the field where I worked for a summer was putting a different propeller on his aircraft. This pilot did not have a basic understanding of the regulations as they applied to a pilot’s ability to perform maintenance. His answer to my question about his authority to replace his propeller was that it was "within his ability." What he knew, but didn't want to divulge was that his abilities had nothing to do with the authority to perform a propeller replacement. First a pilot is limited to preventative maintenance as it is listed in FAR part 43 Appendix A. Furthermore this pilot installed this prop without any guidance or maintenance manual. This is totally wrong. The pilot said that he only needed to make an entry into the logbook stating that this propeller was installed on this airframe. I don't believe the pilot knew the significance of the yellow tag or if the propeller even had one. To this day I still don't know if the pilot even installed a prop that was type certified for the aircraft. This is a case where undocumented or improperly documented changes can impact the safety of flight, and it still happens.
Undocumented Repairs - Former Repair - See Photo
During a prepurchase evaluation I discovered a serious issue with an aircraft's fuselage structure that was undocumented. the aircraft had received minor damage a few month prior and the damage that was recorded was apparently done so that the repairs would not constitute substantial damage in so far as the reporting requirements to the NTSB. What was repaired but not documented to circumvent the required reporting to the NSTB was a replacement of a fuselage former. This former was initially overlooked by me until I had photos developed and I had an opportunity to study the photos of the tailcone section. I suspect the reason this repair was left undocumented is because it would have required a filing of an aircraft accident to the NTSB and filing of a major repair to the FAA which in turn would reduce the value of the aircraft. The repair was done correctly and the aircraft would be worth more if the repair had been documented. Partly because I would not advise my client to purchase this aircraft because of this undocumented repair. Remember "Where there’s smoke there’s fire." If the owner was willing to hide this, what else was he willing to compromise. He obviously doesn't want to fly this airplane anymore. Why else would he be selling it? So be careful when preparing to purchase an aircraft and be sure your mechanic who is reviewing the aircraft is very meticulous. This could be the difference in hundreds of thousands of dollars.
Poor Record Keeping - Keep Your Work Orders
As a matter of civil liability repair stations do not like to keep work orders any longer than they have to. Be ready to question any facilities that do not want to make copies of the work orders for you records. The reason for the record keeping limitations is not to reduce the responsibility and liability of the repair station. it is to reduce the burden of storing and archiving the records associated with the complexity of aircraft maintenance operations. The FARs provide a two year record keeping requirement for the repair stations after which most repair stations prefer to destroy these records on the grounds that the are taking up space when in actuality they are destroying these records often to eliminate any liability. This is an issue that should be addressed by the regulatory authorities and should be considered a top priority. I will hear from the repair station owners on this one, but the reality is if you are providing proper maintenance and recording the maintenance your facility performsScience Articles, then the records would only support your defense. As an owner it is your responsibility to ensure airworthiness. this includes the continuity of your records which is what supports your claims of airworthiness. I would advise to all aircraft owners to make sure that the facilities that provide their maintenance properly and thoroughly document the maintenance that took place on each visit.
ABOUT THE AUTHOR
Grant Wallace founded Analytical Aviation in January 2000. Since that time he has been working toward improving aviation maintenance through technology. He has been the driving force in his comapny's technology products. Mr. Wallace's experience is in aviation maintenance and he has been involved in aviation from a very young age. He has worked from simple line maintenance to part 121 maintenance and developed maintenance programs for regulated operations. He has also managed several corporate aircraft for different companies in the past. His military experience spans from large transport aircraft maintenance to small rotor wing maintenance. He has been a military aircrew member and he holds a degree in industrial engineering technology, a private pilot's certificate with an instrument rating and an A&P. Grant Wallace can be contacted at 1-888-484-4321.
Undocumented damage is most likely the cause of an undocumented repair. The FARs do not discuss undocumented damage. They do however discuss undocumented repairs. These are repairs performed to and aircraft, engine propeller, or appliance that is not documented properly in the maintenance records. There are only a few reasons this would occur. We will discuss the reasons in this month's newsletter.
Unintentional Damage Examples
Hanger Rash
Hanger Rash is a common undocumented damage. Which can easily lead to an undocumented repair. Let’s say a lineman was re-positioning aircraft to get ready for the next day's departures when he accidentally clips a position light against the hanger door. He realizes this and sees that it pulled the light out from the wings fiberglass mounting surface just slightly. He, who is mechanically inclined but not authorized to do a repair, gets his tool box and removes the wingtip and institutes a substandard and illegal repair to prevent anyone from realizing that any damage occurred. Now he believes he has done the right thing, after all he has saved his company from the time and embarrassment associated with dealing with this problem and the customer can fly his plane the next morning. This is obviously a rare occurrence but none the less it happens all too often. This is an example where undocumented damage becomes an undocumented repair.
Bird Strike
Bird Strikes are even less common but do pose a real issue with undocumented damage. A pilot is on short final when a small flock of birds is startled and fly into his path. He receives a strike against his wing's leading edge and it dents slightly. This pilot who is an A&P looks at the damage and decides that it is negligible and does not report the damage to the owner. This bird strike was later seen by the owner and he asked the mechanic about it. The mechanic told him the story and said it was a negligible dent and wouldn't affect the flight characteristics of the aircraft. And besides "you don't want a damage history". Well this is entirely all too common. A mechanic advises his client to not document or repair negligible damage due to its effect on aircraft value. This is simply not good advice. At the end of the day the aircraft must conform to its original type design and any negligible damage is not part of the type design. I know what you’re thinking and the answer is "It is a very narrow line to walk but use your judgment on what should be repaired and what really is negligible." Regardless it must all be documented.
Hard Landings
This is common among training aircraft. The reality is that aircraft are very very tough. Sometime though, not tough enough. Some of the hidden damage could be overlooked for several inspection cycles and sometime maybe not discovered at all. It is important to make sure that when you experience a hard landing that you have the aircraft thoroughly inspected SPECIFICALLY for that damage. Do not wait until the next 100 hour and say "Oh yeah, I had a student really drop this one on its nose last month. Could you look for any damage?" What happens here is that the mechanic is performing a 100 hour inspection, not a hard landing inspection. The inspection should always equal the suspected damage. This is why when you suspect any damage you should inspect for that damage immediately. Don't put it off just because you don't see any obvious defects.
Un-Ethical Practices To Reduce Damage History
Believe it or not it happens. This is when a person provides substandard services, illegal repairs, when an owner "Fixes his plane", and when a person puts the future aircraft value ahead of safety and the rule of regulatory compliance. Where the events and scenarios above were fictitious these will be directly from my experience as a mechanic.
Cowling Alteration - Substandard Service
I was working at a facility where a light twin had been grounded for being un airworthy. The aircraft had some major issues but could be easily repaired. One of the damaged components was the aft mounting plates for the engine cowlings. The repair was completed and the mechanics moved on to the next issue. During this maintenance the aircraft received some hanger rash and a new elevator was required. This became the major issue and once resolved the aircraft was signed off as airworthy but the repair to the cowling was never documented. The service the owner received was not to the standards required by the FARs and the owner should have expected these mechanics to do a better job. Be very careful when hiring mechanics as they are not all equal.
Ice Shield Repair - Illegal Repairs
When I worked for an airline I was involved in a squawk that required a repair or replacement of a fuselage ice shield. The replacement part was in the computer inventory and I removed the damaged component. The question that was raised in this case is what happens if the replacement part is not actually in stock? Well unfortunately the broad answer is not so easy. My answer is you cannot reinstall that damaged item even if it was deferrable due to the fact that the steps in properly documenting a maintenance event is to record the removal of the damaged part. Once you have committed to removing the damaged part it becomes un-airworthy and cannot be re-installed regardless of its deferability in the MEL.
The actual events that occurred were as follows:
I noticed a significant impact to the ice shield and wrote it up as being damaged.
The inspector told me to replace the ice shield.
I confirmed the availability of the replacement part in the computer inventory.
I removed the damaged part and prepared the surface for the replacement part.
I sent for the new part and was told that we didn't have it in stock. (Inventory Error)
The Inspector said to re-install the old one.
I refused.
He said to repair it.
I researched the SRM and found that we did not have the proper facility to repair it.
He said to just put some 5 minute epoxy in the area and sand it down and paint it.
I walked away.
He found someone else to jeopardize their standards and the aircraft departed on time, illegal, and unairworthy.
This is a common practice and is overlooked by many people and can be considered as SOP for some maintenance facilities. It is the responsibility of the mechanic to stand firm on his knowledge of the regulations and question the actions of his supervisors if the policy is to circumvent the rules to make an on time departure.
Owner Repairs - Propeller Installation - Improper Authority
This is a case of blatant regulatory violation. A pilot who tied his aircraft down at the field where I worked for a summer was putting a different propeller on his aircraft. This pilot did not have a basic understanding of the regulations as they applied to a pilot’s ability to perform maintenance. His answer to my question about his authority to replace his propeller was that it was "within his ability." What he knew, but didn't want to divulge was that his abilities had nothing to do with the authority to perform a propeller replacement. First a pilot is limited to preventative maintenance as it is listed in FAR part 43 Appendix A. Furthermore this pilot installed this prop without any guidance or maintenance manual. This is totally wrong. The pilot said that he only needed to make an entry into the logbook stating that this propeller was installed on this airframe. I don't believe the pilot knew the significance of the yellow tag or if the propeller even had one. To this day I still don't know if the pilot even installed a prop that was type certified for the aircraft. This is a case where undocumented or improperly documented changes can impact the safety of flight, and it still happens.
Undocumented Repairs - Former Repair - See Photo
During a prepurchase evaluation I discovered a serious issue with an aircraft's fuselage structure that was undocumented. the aircraft had received minor damage a few month prior and the damage that was recorded was apparently done so that the repairs would not constitute substantial damage in so far as the reporting requirements to the NTSB. What was repaired but not documented to circumvent the required reporting to the NSTB was a replacement of a fuselage former. This former was initially overlooked by me until I had photos developed and I had an opportunity to study the photos of the tailcone section. I suspect the reason this repair was left undocumented is because it would have required a filing of an aircraft accident to the NTSB and filing of a major repair to the FAA which in turn would reduce the value of the aircraft. The repair was done correctly and the aircraft would be worth more if the repair had been documented. Partly because I would not advise my client to purchase this aircraft because of this undocumented repair. Remember "Where there’s smoke there’s fire." If the owner was willing to hide this, what else was he willing to compromise. He obviously doesn't want to fly this airplane anymore. Why else would he be selling it? So be careful when preparing to purchase an aircraft and be sure your mechanic who is reviewing the aircraft is very meticulous. This could be the difference in hundreds of thousands of dollars.
Poor Record Keeping - Keep Your Work Orders
As a matter of civil liability repair stations do not like to keep work orders any longer than they have to. Be ready to question any facilities that do not want to make copies of the work orders for you records. The reason for the record keeping limitations is not to reduce the responsibility and liability of the repair station. it is to reduce the burden of storing and archiving the records associated with the complexity of aircraft maintenance operations. The FARs provide a two year record keeping requirement for the repair stations after which most repair stations prefer to destroy these records on the grounds that the are taking up space when in actuality they are destroying these records often to eliminate any liability. This is an issue that should be addressed by the regulatory authorities and should be considered a top priority. I will hear from the repair station owners on this one, but the reality is if you are providing proper maintenance and recording the maintenance your facility performsScience Articles, then the records would only support your defense. As an owner it is your responsibility to ensure airworthiness. this includes the continuity of your records which is what supports your claims of airworthiness. I would advise to all aircraft owners to make sure that the facilities that provide their maintenance properly and thoroughly document the maintenance that took place on each visit.
ABOUT THE AUTHOR
Grant Wallace founded Analytical Aviation in January 2000. Since that time he has been working toward improving aviation maintenance through technology. He has been the driving force in his comapny's technology products. Mr. Wallace's experience is in aviation maintenance and he has been involved in aviation from a very young age. He has worked from simple line maintenance to part 121 maintenance and developed maintenance programs for regulated operations. He has also managed several corporate aircraft for different companies in the past. His military experience spans from large transport aircraft maintenance to small rotor wing maintenance. He has been a military aircrew member and he holds a degree in industrial engineering technology, a private pilot's certificate with an instrument rating and an A&P. Grant Wallace can be contacted at 1-888-484-4321.
Subscribe to:
Posts (Atom)