Mercado de oferta

In this table, outliers dataset has been included as it was meant or an anomaly detection algorithm. Performance results are as follows.

Twitter’s Anomaly Detection Dataset Threshold

(%)

Accuracy Error rate Precision Recall Specificity F-measure

Outliers 1,25,49 0.994312 0.005687 1 0.09 1 0.165137

Sine Wave 1,25,49 0.985714 0.014285 N/A 0 1 N/A

Stationary 1,25,49 0.993752 0.006252 N/A 0 1 N/A

Synthetic 1 1,25,49 0.997619 0.002380 1 0.555555 1 0.714285

Synthetic 2 1,25,49 0.998212 0.001785 1 0.666666 1 0.8

Real Yahoo Data 1 0.997946 0.002053 1 0.8125 1 0.896551 25, 49 0.988497 0.011502 0.4848 1 0.988 0.653061

Figure 37 shows the average precision, recall and f-measure of Pelt and Prophet on every dataset except non-stationary data.

Figure 37: Average precision, recall and f-measure of Prophet and PELT on 5 datasets, except non-stationary data

From the plot, it can be seen that PELT is the clear winner when it comes to all the metrics. It means that it detects true changepoints far better than Prophet, and detects less false negatives than prophet.

Figure 38 shows the average precision, recall and f-measure of Pelt and Prophet on non-stationary data.

Figure 38: Average precision, recall and f-measure of Prophet and PELT on non-stationary data

From the plot it can be seen that PELT is better than Prophet on non- stationary data. The precision is almost the same in both cases, but recall is where PELT performs better. This shows that PELT detects far less false negatives than Prophet.

Figure 39 is the plot of precision of Twitter’s anomaly detection over all the datasets. Precision for sine wave and stationary data were not available, so they have not been included in the plot.

Figure 39: Precision of Twitter’s AD on outlier, synthetic 1, synthetic 2 and real datasets.

From the plot it can be seen that the precision is a 100% in the case of outliers data, and synthetic data 1 and 2. No false positives were detected, and points which in ground truth were anomalous have been detected as anomalies. Twitter does good on real data as well, with a precision of 75%.

Figure 40 is a plot of recall of Twitter’s anomaly detection for all datasets except sine curve and stationary data, because this metric for those 2 datasets are not available.

Figure 40: Recall of Twitter’s anomaly detection for outliers, synthetic data 1, synthetic data 2 and real world data

From the graph it can be seen that recall is highest in the case of real world data, which means no false negatives were present. Recall is lowest in the case of outliers data, which means lot of false negatives were present. Synthetic datasets both have a recall of 0.5 and 0.6, which means the performance of the algorithm was mediocre, as many false negatives were present as true positives.

Figure 41 is an image of F-measure of Twitter’s anomaly detection, the data for sine wave and stationary signal was not available, so they have not been included in the plot.

Figure 41: F-meausre of Twitter’s AD on outlier, synthetic 1, synthetic 2 and real datasets

From the plot it can be seen that the F-measure is almost the same for synthetic and real datasets, and the value is around the same, also, pretty high. Given that precision and recall for these datasets was high, it is natural F- measure is high as well, since F-measure is dependent on precision and recall. This metric is low for outliers dataset, given the recall for outliers dataset was really low, it dragged the F-measure to a low value.

Run time of algorithms was also taken into account. The algorithms were run on a local machine with an Intel i5 CPU with 4 cores clocked at 2.4 GHz, 8 gigabytes of RAM, Windows 10 enterprise operating system system, and Intel HD graphics with 128 MB of memory.

Figure 42 is a plot of runtime of Prophet on all datasets with increasing number of points. Time is in seconds.

Figure 42: Run time of Prophet on datasets with increasing number of points Figure 43 is a plot of runtime of PELT on all datasets with increasing number of points. Time in seconds.

Figure 43: Runtime of PELT on datasets with increasing number of points Figure 44 is a plot of runtime of Twitter’s Anomaly detection on all datasets with increasing number of points.

Figure 44: Runtime of Twitter’s anomaly detection on all datasets with increas- ing number of points

From the plots it can be seen that the runtime of algorithms increases with increase in data points. Run time of Twitter’s AD is 10 times more than run time of PELT in case of Yahoo synthetic datasets, and run time of Prophet is two times the run times the run time of Twitter’s AD for the same datasets. When datasets with more than 30,000 points are considered (change in mean, non-stationary data in case of changepoint data, stationary and outlier data in case of anomaly detection data), Prophet is the slowest, taking more than 100 seconds, Twitter’s AD is second slowest, taking an average of 45 seconds, and PELT is the fastest, taking just 0.2 seconds. Overall, PELT is the fastest algorithm among the three.