Data Mining and Data warehousing,Unit IV ,Two marks questions with answers

Data Mining and Data warehousing   

Two marks questions with answers

Unit IV

1)      Define classification.

Data classification is a two step process. In the first step a model is built describing a predetermined set of data classes are concepts. The model is constructed by analyzing database tuples described b attributes. Each tuple is assumed to belong to the predefined class as determined by one of the attributes called class label attribute. In the second step the modal is used for classification.

2)      Define training data set.

The data tuples analyzed to build the model collectively form the training data set. Individual tuples making up the training set are referred to as training samples and a randomly selected from the sample population.

3)      Define accuracy of a model.

The accuracy of a model on a given test set is the percentage of test set samples that are correctly classified by the model.

4)      Define prediction.

Prediction can be viewed as the construction and use of a model to assess the class of an unlabeled sample or to assess the value or value ranges of an attribute that a given sample is likely to have. Classification and regression are the two major types of prediction.

5)      Differentiate classification and prediction.

Classification is used to predict discrete or nominal values whereas prediction is used to predict continuous values. Classification is also known as supervised learning whereas prediction is also known as unsupervised learning.

6)      List the applications of classification and prediction.

Applications include credit approval, medical diagnosis, performance prediction, and selective marketing.

7)      List the preprocessing steps involved in preparing the data for classification and prediction.

Data cleaning, Relevance analysis, data transformation.

8)      Define normalization.

Normalization involves scaling all values for a given attribute so that they fall within a small specified range such as -1.0 to 1.0 or 0.0 to 1.0.

9)      What is a decision tree?

A decision tree is a flow chart like structure, where each internal node denotes a test on an attribute, each branch represents an outcome of the test, and leaf nodes represent classes or class distribution. The top most node in the tree is called root node.

10)  Define tree pruning.

When decision trees are built many of the branches may reflect noise or outliers in training data. Tree pruning attempts to identify and remove such branches with the goal of improving classification accuracy on unseen data.

11)  Define information gain.

The information gain measure is used to select the test attribute at each node in the tree. Such a measure is referred to as an attribute selection measure or a measure of the goodness of split. The attribute with the highest information gain is chosen as the test attribute for the current node. This attribute minimizes the information needed to classify the samples in the resulting partitions and reflects the least randomness is “impurity” in the partitions.

12)   List the two common approaches for tree pruning.

Prepruning approach – a tree is “Pruned” by halting its construction early. Upon halting the node becomes a leaf. The leaf may hold the most frequent class among the subsets samples or the probability distribution of the samples.

Post pruning approach – removes branches from a “fully grown” tree. A tree node is pruned by removing its branches the lowest unpruned node becomes the leaf and is labeled by the most frequent class among its former branches.

13)   List the problems in decision tree induction and how it can be prevented.

Fragmentation, repetition, and replication. Attribute construction is an approach for preventing these problems, where the limited representation of the given attributes is improved by creating new attributes based on the existing ones.

14)   What are Bayesian classifiers?

Bayesian classifiers are statistical classifiers. They can predict class membership probabilities, such as the probability that a given sample belongs to a particular class. Bayesian classification is based on bayes theorem. Bayesian classifiers exhibit high accuracy and speed when applied to large databases. Bayesian classifier also known as naïve Bayesian classifiers is comparable in performance with decision tree and neural network classifiers.

15)   Define Bayesian belief networks.

Bayesian belief networks are graphical models which allow the representation of dependencies among subsets of attributes. It can also be used for classification.

16)   List the two components in belief network.

Directed acyclic graph, conditional probability table. 

17)   Define directed acyclic graph.

In Directed Acyclic graph each node represents a random variable and each arch represents a probabilistic dependence. If an arch is drawn from a node Y to a node Z then Y is a parent or immediate predecessor of Z and Z is a descendent of Y. Each variable is conditionally independent of its non-descendents in the graph given its parents. The variables may be discrete or continuous – valued.

18)   Define Conditional Probability Table (CPT).

The CPT for a variable Z specifies the conditional distribution

P (Z| parents (Z)), where parents (Z) are the parents of Z.

19)   List the methods used for classification based on concepts from association rule mining.

ARCS (Association Rule Clustering System), Associative classification, CAEP (Classification by Aggregating Emerging Patterns).

20)   Explain ARCS?

ARCS mines association rules of the form A_quan1 ^ A_quan2 => A_cat where A_quan1 and A_quan2 are tests on quantitative attribute ranges and A_cat assigns a class label for a categorical attribute from the given training data. Association rules are plotted on a 2D grid. The algorithm scans the grid, searching for rectangular clusters of rules. Adjacent ranges are the quantitative attributes occurring within a rule cluster may be combined. The clustered association rules generated by ARCS are applied to classification.

Data Mining and Data warehousing,Unit III ,Two marks questions with answers

Data Mining and Data warehousing   

Two marks questions with answers 

Unit III

 1)      Define support in association rule mining

The rule A => B holds in the transaction set D with support s where s is the percentage of transactions in D that contain A U B i.e., both A & B. This is taken to be the probability, P (A U B).

2)      Define confidence.

The rule A => B has confidence c in the transaction set D if c is the percentage of transactions in D containing A that also contains B. This is taken to be the Conditional Probability P (B|A).

3)      Define occurrence frequency of an item set.

A set of items is referred to as an item set. The occurrence frequency of an item set is the number of transactions that contain the item set.

4)      How is association rules mined from large databases?

Association rule mining is a two step process.

Find all frequent item sets

Generate strong association rules from the frequent item sets

5)      When an item set satisfies minimum support?

An item set satisfies minimum support if the occurrence frequency of the item set is greater than or equal to the product of min_sup and the total number of transactions in D.

6)      Define minimum support count.

The number of transactions required for the item set to satisfy minimum support is therefore referred to as minimum support count. If an item set satisfies minimum support then it is a frequent item set.

7)      Give the classification of association rules.

Based on the types of values handled in the rule

Based on the dimensions of data involved in the rule.

Based on the levels of abstractions involved in the rule set.

Based on various extensions to association mining.

8)      Define Frequent Closed Item Set.

Frequent Closed Item Set is a frequent closed item set where an item set c is closed if the there exists no proper superset of c, c’ such that every transaction containing c also contains c’.

9)      Define Apriori property.

If an item set I does not satisfy the minimum support threshold, min_sup then I is not frequent i.e., P (I) < min_sup. If an item A is added to the item set I then the resulting item set I U A cannot occur more frequently than I. Therefore I U A is not frequent either i.e., P (I U A) < min_sup. 

10)   Define Anti-Monotone property.

If a set cannot pass a test or, all of its supersets will fail the same test as well. It is called anti-monotone because property is monotonic in the context of failing a test.

11)   List the two step process involved in Apriori algorithm.

Join Step

Prune Step

12)   List the search strategy for mining multi level associations with reduced support.

Level by level independent

Level cross filtering by single item

Level cross filtering by K item set.

13)   Compare Level by level independent and level cross filtering by K item set strategy.

Level by level independent strategy lead to examining numerous infrequent at low levels finding association between items of little importance.

Level cross filtering by K item set strategy allows the mining system to examine only the children of frequent K item sets. This restriction is very strong in that, there usually are not K item sets that when combined are also frequent. Hence many valuable patterns may be filtered out using this approach.

14)   Define single dimensional association rule.

Buys(X, “IBM desktop computer”) => buys(X, “Sony b/w printer”)

The above rule is said to be single dimensional rule since it contains a single distinct predicate (eg buys) with multiple occurrences (i.e., the predicate occurs more than once within the rule. It is also known as intra dimension association rule.  

15)   Define multi dimensional association rules.

Association rules that involve two or more dimensions or predicates can be referred to as multi dimensional associational rules.

Age(X, “20…29”) ^ occupation (X, “Student”) => buys (X,”Laptop”)

The above rule contains three predicates (age, occupation, buys) each of which occurs only once in the rule. There are no repeated predicates in the above rule. Multi dimensional association rules with no repeated predicates are called interdimension association rules. 

16)   Define categorical attribute.

Categorical attributes have finite number of possible values with no ordering among the values (eg. Occupation, brand). Categorical attributes are also called nominal attributes since there values are “Names of Things”. 

17)   Define Quantitative Attributes.

Quantitative attributes are numeric and have an implicit ordering among values (eg age, income, and price).

Data Mining and Data warehousing,Unit II,Two marks questions with answers

Data Mining and Data warehousing   

Two marks questions with answers

Unit II

1. Why preprocess the data?

Data that is to be analyzed by data mining techniques are incomplete, noisy, and inconsistent. These are the commonplace properties of large real world databases and data warehouses. To remove all these errors data must be preprocessed.

2. What are the data preprocessing techniques?

Data preprocessing techniques are

Data cleaning-removes noise and correct inconsistencies in the data.

Data integration-merges data from multiple sources into a coherent data store such as data warehouse or a data cube.

Data transformations-such as normalization improve the accuracy and efficiency of mining algorithms involving distance measurements.

Data reduction-reduces the data size by aggregating, eliminating redundant features, or clustering

3. Give the strategies involved in data reduction.

Data reduction obtains a reduced representation of the data set that is much smaller in volume but produces the same analytical results. The strategies involved are:

Data aggregation-eg building a data cube.

Dimension reduction-eg removing irrelevant attributes through correlation analysis.

Data compression-eg using encoding schemes such as minimum length encoding or wavelets

4. What is noise?

Noise is a random error or variance in a measured variable.

5. Give the various data smoothing techniques.

Binning, clustering, combined computer and human inspection, regression.

6. Define data integration.

Data integration combines data from multiple sources into a coherent data store. These sources may include multiple databases, data cubes or flat files.

7. Give the issues to be considered during data integration.

Schema integration, Redundancy, detection and resolution of data value conflicts.

8. Define redundancy.

An attribute is said to be redundant if it can be derived from another table. Inconsistencies in attribute or dimension naming can also cause redundancies in the resulting data set.

9. Explain data transformation.

In data transformation data are transformed or consolidated into forms appropriate for mining. Data transformation can involve the following, smoothing, aggregation, generalization, normalization, and attribute construction.

10. Define normalization and give the methods used for normalization.

In normalization attribute data are scaled so as to fall within a small specified range such as -1.0 to 1.0 or 0.0 to 1.0. The methods used for normalization are min-max normalization, z-score normalization and normalization by decimal scaling.

11. Define data reduction.

It is used to obtain a reduced representation of the data set that is much smaller in volume yet closely maintains the integrity of the original data. I-e mining on the reduced set should be more efficient yet produce the same analytical results.

12.  Give the strategies used for data reduction.

Data cube aggregation, dimension reduction, data compression, numerosity reduction, and discretization and concept hierarchy generation.

13. Explain data cube aggregation.

Data cube store multidimensional aggregated information. Each cell holds an aggregate data value data value, corresponding to the data point in multidimensional space.

14. Explain dimensionality reduction.

Data sets for analysis may contain hundreds of attributes many of which may be irrelevant to the mining task, or redundant. Leaving out relevant attributes or keeping irrelevant attributes may result in poor quality of discovered patterns.

15. How dimensionality reduction is achieved?

Dimensionality reduction is achieved using attribute subset selection. The goal is to find a minimum set of attributes such that the resulting probability distribution of the data classes is as close as possible to the original distribution obtained using all attributes.

16.  Define data compression.

In data compression, data encoding or transformation are applied so as to obtain a reduced or “compressed” representation of the original data. If original data can be reconstructed from the compressed data without any loss of information, data compression technique used is called lossless. If only an approximation of the original data is obtained then it is called lossy.

17. Give examples for lossy data compression.

Wavelet transforms and principal component analysis.

18. Define Discrete Wavelet transforms.

DWT is a linear signal processing technique that when applied to a data vector D, transforms it to a numerically different vector, D’ of wavelet coefficients. The two vectors are of the same length. It achieves better lossy compression.

19. Define numerosity reduction.

Numerosity reduction is a technique used to reduce the data volume by choosing alternative smaller forms of data representation. These techniques may be parametric or nonparametric.

20. Give examples for parametric and nonparametric methods.

Parametric-Log-linear models

Non-parametric-histogram, clustering, sampling