Mineração em Data Streams - Weka e MOA. Profa. Elaine Faria UFU

Transcrição

1 Mineração em Data Streams - Weka e MOA Profa. Elaine Faria UFU

2 O que é o Weka? Ferramenta para Aprendizado de Máquina desenvolvida em Java Criado por pesquisadores da Universidade de Waikato Nova Zelândia Implementa inúmeras funcionalidades Classificadores Métodos de agrupamento Técnicas de pré-processamento

3 Weka Operações Usando interface gráfica Usando linha de comando JAVA API Para download, acesse: Você pode fazer o download da versão executável e dos códigos fontes

4 Weka Formato dos dados Usa arquivos textos para descrever os dados Trabalha com uma grande variedade de tipos de arquivos, incluindo.arff É possível importar dados CSV, por exemplo Dados podem ser lidos de uma URL ou de uma base SQL

5 Formato dos sepallength sepalwidth petallength petalwidth class {Iris-setosa,Iris-versicolor, 5.1,3.5,1.4,0.2,Iris-setosa 4.9,3.0,1.4,0.2,Iris-setosa 4.7,3.2,1.3,0.2,Iris-setosa

6 Formato dos attrname {numeric, string, <nominal>, date} Numeric: número Nominal: conjunto finito de strings String: concatenação de caracteres Date: data formato yyyy-mm-dd

7 Formato dos age sex { female, chest_pain_type { typ_angina, asympt, non_anginal, cholesterol exercise_induced_angina { no, class { present, 63,male,typ_angina,233,no,not_present 67,male,asympt,286,yes,present 67,male,asympt,229,yes,present 38,female,non_anginal,?,no,not_present

8 Weka

9 Weka

10 Pré-processamento É chamado no Weka de filtro (filters) Weka contém filtros Discretização, normalização, reamostragem, seleção de atributos, etc.

11 Exemplo de Pré-processamento Remove RemoveWithValues Discretization NominalToBinary Ver vídeo unscy

12 Visualize

13 Vídeos sobre o Weka

14 MOA MOA (Massive On-line Analysis) is a framework for data stream mining. It includes tools for evaluation and a collection of machine learning algorithms. Related to the WEKA project, it is also written in Java. The goal of MOA is a benchmark framework for running experiments in the data stream mining context Retirado de:

15 MOA Contains storable settings for data streams (real and synthetic) for repeatable experiments a set of existing algorithms and measures form the literature for comparison and an easily extendable framework for new streams, algorithms and evaluation methods. Retirado de:

16 MOA Using MOA first a data stream (feed, generator) is chosen and configured second an algorithm (e.g. a classifier) is chosen and its paramters are set third the evaluation method or measure is chosen and finally the results are obtained after running the task Retirado de:

17 MOA To run an experiment using MOA, the user can choose between a graphical user interface (GUI) or a command line execution Retirado de:

18 MOA MOA currently supports stream classification, stream clustering, outlier detection, change detection and concept drift and recommender systems. They are working on extending MOA to support other mining tasks on data streams. Retirado de:

19 MOA - Stream Classification Data Sources or Streams: ARFF Reader, Random Tree Generator, SEA Concepts Generator, STAGGER Concepts Generator, Rotating Hyperplane, Random RBF Generator, LED Generator, Waveform Generator, and Function Generator. Classifiers: Naive Bayes, Hoeffding Tree, Hoeffding Option Tree, Hoeffding Adaptive Tree, Bagging, Boosting, Bagging using ADWIN, Leveraging Bagging, SGD, Perceptron, SPegasos. Evaluation procedures for Data Streams: Holdout and Interleaved Test-Then-Train or Prequential Retirado de:

20 MOA - Stream Clustering Generators: support cluster evolution events (merging or disappearing), configuration of dimensionality, number and size of clusters, drift speed, decay horizon (aging) and noise rate etc. Algorithms: StreamKM++, CluStream, ClusTree, DenStream, D-Stream, CobWeb Evaluation: Internal and External Retirado de:

21 Extending MOA To add a new stream classifier algorithm, implement the Classifier.java interface with the following three main methods void resetlearningimpl(): a method for initializing a classifier learner void trainoninstanceimpl(instance): a method to train a new instance double[] getvotesforinstance(instance): a method to obtain the prediction result Retirado de:

22 Extending MOA To add a new stream clustering algorithm, implement the Clusterer.java interface with the following three main methods void resetlearningimpl(): a method for initializing a clusterer learner void trainoninstanceimpl(instance): a method to train a new instance Clustering getclusteringresult(): a method to obtain the current clustering result for evaluation or visualization

23 Repositório de Dados UCI Activity recognition Challenge

24 Referências ial.ppt