Adaptive stratified sampling to support high uniformity confidence in data stream environment

초록

In this paper, we focus on improving the accuracy of UC-KSample, which has random sampling characteristics in stream data and at the same time supports high uniformity confidence. To do this, we propose Stratified UC-KSample (S-UCKSample in short) by applying UC-KSample to stratified sampling. In this process, we need to support two features: (1) constant sampling rate and (2) constant uniformity confidence. If we simply apply UC-KSample to stratified sampling, the constant sampling rate is satisfied, but the constant uniformity confidence is not satisfied. We call this an unstable uniformity confidence problem and present the concept of window restart to solve the problem. The window restart makes all substreams create new windows if the uniformity confidence of any substream exceeds the given threshold. Using this window restart, we can always maintain the uniformity confidence above the given threshold. Experimental results show that the proposed S-UCKSample improves the accuracy more than 10 times compared with the basic UC-KSample. © 2018 ICIC International.

키워드