Hi, when the split is set to 50% normal. The y_test is generated such a way the anomalies are always grouped.
Just use this command " print(np.where(y_test==1)) " and you can see the problem.
For example, this is the result of above command when applied on '01-thyroid.csv'
(array([1840, 1841, 1842, 1843, 1844, 1845, 1846, 1847, 1848, 1849, 1850,
1851, 1852, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1860, 1861,
1862, 1863, 1864, 1865, 1866, 1867, 1868, 1869, 1870, 1871, 1872,
1873, 1874, 1875, 1876, 1877, 1878, 1879, 1880, 1881, 1882, 1883,
1884, 1885, 1886, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894,
1895, 1896, 1897, 1898, 1899, 1900, 1901, 1902, 1903, 1904, 1905,
1906, 1907, 1908, 1909, 1910, 1911, 1912, 1913, 1914, 1915, 1916,
1917, 1918, 1919, 1920, 1921, 1922, 1923, 1924, 1925, 1926, 1927,
1928, 1929, 1930, 1931, 1932]),)
Due to this grouping in the testdata we are getting good results. When we shuflle x_test and y_test externally and apply any algorithm the results are poor.
Please remove the bias in test data splitting. Shuffle the anomalies.