Comments for Data Mining Research http://www.dataminingresearch.com Data Mining Research, Algorithms, Tools, News, More Thu, 06 Oct 2011 20:24:21 +0000 hourly 1 http://wordpress.org/?v=3.0 Comment on Classic3 and Classic4 DataSets by Volkan TUNALI http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-321 Volkan TUNALI Thu, 06 Oct 2011 20:24:21 +0000 http://www.dataminingresearch.com/?p=184#comment-321 Amara, As far as I know, there is no query set for these datasets. Given matrix files are only for document collections, so they do not include any query-spesific data. I use the software tools that I develop for preprocessing the original text document collections. Currently I cannot share the software. I'm planning to share the software after I complete my PhD thesis, possibly within 6 months or so. I'm sorry. I think you can find several text preprocessing tools on the web. Simply, the software counts the each word in each document; after getting the word frequencies, those frequencies are weighted according to TFIDF scheme. That's all. I hope this information will help you. Good luck. Amara,

As far as I know, there is no query set for these datasets. Given matrix files are only for document collections, so they do not include any query-spesific data.

I use the software tools that I develop for preprocessing the original text document collections. Currently I cannot share the software. I’m planning to share the software after I complete my PhD thesis, possibly within 6 months or so. I’m sorry. I think you can find several text preprocessing tools on the web. Simply, the software counts the each word in each document; after getting the word frequencies, those frequencies are weighted according to TFIDF scheme. That’s all. I hope this information will help you. Good luck.

]]> Comment on Classic3 and Classic4 DataSets by Amara http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-320 Amara Thu, 06 Oct 2011 15:33:24 +0000 http://www.dataminingresearch.com/?p=184#comment-320 As i understand, these datasets are IR datasets. They have their own query sets aswell. (then you can match or rank documents for each query. similar to document retrieval). Are the query sets included in your preprocessed collection? if noet , then can you please share the software/technique you have used to pre-process documents so that query set can be processed the same way! As i understand, these datasets are IR datasets. They have their own query sets aswell. (then you can match or rank documents for each query. similar to document retrieval). Are the query sets included in your preprocessed collection? if noet , then can you please share the software/technique you have used to pre-process documents so that query set can be processed the same way!

]]> Comment on Classic3 and Classic4 DataSets by Volkan TUNALI http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-313 Volkan TUNALI Thu, 04 Aug 2011 16:49:10 +0000 http://www.dataminingresearch.com/?p=184#comment-313 Mehrdad, I usually cite Classic datasets like below: Classic3 dataset. Retrieved November 29, 2009 from World Wide Web: ftp://ftp.cs.cornell.edu/pub/smart If you need to cite me due to the preprocessed dataset files, I think you may cite similarly like: Classic3 dataset. Retrieved November 29, 2009 from World Wide Web: http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets Thanks. Mehrdad,

I usually cite Classic datasets like below:

Classic3 dataset. Retrieved November 29, 2009 from World Wide Web: ftp://ftp.cs.cornell.edu/pub/smart

If you need to cite me due to the preprocessed dataset files, I think you may cite similarly like:

Classic3 dataset. Retrieved November 29, 2009 from World Wide Web: http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets

Thanks.

]]> Comment on Classic3 and Classic4 DataSets by mehrdad http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-312 mehrdad Thu, 04 Aug 2011 14:58:20 +0000 http://www.dataminingresearch.com/?p=184#comment-312 Hi, I'm using the processed Classic dataset in my paper. I'd like to konw how can I site you? Regards, Mehrdad Hi,
I’m using the processed Classic dataset in my paper. I’d like to konw how can I site you?
Regards,
Mehrdad

]]> Comment on Classic3 and Classic4 DataSets by newbi http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-259 newbi Fri, 13 May 2011 03:18:28 +0000 http://www.dataminingresearch.com/?p=184#comment-259 Sorry, i explained it not clearly last time. Some papers explain that for their experiment, from each class they prepared a kind of sub-classes. For example take one: the CACM, they have this statistic (only example): Num of Doc : 500 Num of class : 4 Min class size : 80 Max class size : 120 Avg Doc Length : 100 Etc. Could you please explain where those statistics came from? Are they provided in Classic4 or decided by ourselves. ThankYou. Sorry, i explained it not clearly last time.
Some papers explain that for their experiment, from each class they prepared a kind of sub-classes. For example take one: the CACM, they have this statistic (only example):
Num of Doc : 500
Num of class : 4
Min class size : 80
Max class size : 120
Avg Doc Length : 100
Etc.
Could you please explain where those statistics came from? Are they provided in Classic4 or decided by ourselves.

ThankYou.

]]> Comment on Classic3 and Classic4 DataSets by Volkan TUNALI http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-258 Volkan TUNALI Mon, 09 May 2011 08:54:55 +0000 http://www.dataminingresearch.com/?p=184#comment-258 For the benchmark datasets, we already know the class distribution of the dataset. For example, Classic4 dataset is comprised of documents from 4 classes as: CACM: 3204 documents CISI: 1460 documents CRAN: 1398 documents MED: 1033 documents Is your question something different? For the benchmark datasets, we already know the class distribution of the dataset. For example, Classic4 dataset is comprised of documents from 4 classes as:

CACM: 3204 documents
CISI: 1460 documents
CRAN: 1398 documents
MED: 1033 documents

Is your question something different?

]]> Comment on Classic3 and Classic4 DataSets by newbi http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-257 newbi Mon, 09 May 2011 08:47:32 +0000 http://www.dataminingresearch.com/?p=184#comment-257 Hi Volkan. In some papers using classic4 as dataset, they describe about the class inside dataset. For example in Med and Cisi, there are 4 classes. And the papers also shown the document size of each classes. My question in how to determine the class and the others statistics of a dataset. Because I didnt see any information about the them inside Classic4 datasets. Thank You. Hi Volkan.
In some papers using classic4 as dataset, they describe about the class inside dataset. For example in Med and Cisi, there are 4 classes. And the papers also shown the document size of each classes. My question in how to determine the class and the others statistics of a dataset. Because I didnt see any information about the them inside Classic4 datasets.
Thank You.

]]> Comment on Classic3 and Classic4 DataSets by Volkan TUNALI http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-248 Volkan TUNALI Sat, 02 Apr 2011 21:08:40 +0000 http://www.dataminingresearch.com/?p=184#comment-248 I think you should see my post on "awk" where I give the awk code to split Classic collection into single documents: http://www.dataminingresearch.com/index.php/2010/10/absolute-beginners-first-awk-program/ If it is not what you want, please let me know and please be specific. I think you should see my post on “awk” where I give the awk code to split Classic collection into single documents: http://www.dataminingresearch.com/index.php/2010/10/absolute-beginners-first-awk-program/

If it is not what you want, please let me know and please be specific.

]]> Comment on Classic3 and Classic4 DataSets by Anonymouse http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-247 Anonymouse Sat, 02 Apr 2011 20:47:14 +0000 http://www.dataminingresearch.com/?p=184#comment-247 Hello Volkan, is there any chance you could share the method used to split the files (source code)? Hello Volkan, is there any chance you could share the method used to split the files (source code)?

]]> Comment on Classic3 and Classic4 DataSets by Volkan TUNALI http://www.dataminingresearch.com/index.php/2010/09/classic3-classic4-datasets/#comment-246 Volkan TUNALI Fri, 01 Apr 2011 07:38:54 +0000 http://www.dataminingresearch.com/?p=184#comment-246 Cedric, This is the whole dataset, there is no separate data for training and testing. Figures on the first line means: * There are 7095 documents, * 5896 terms, * and totally 247158 non-zero values within the document-term matrix. The followin lines are the actual non-zero values, with the format of [Document No] [Term No] [Weight]. I hope my explanation make it clear for you. This is a common format, that's why I have not seen a need for explanation. Cedric,

This is the whole dataset, there is no separate data for training and testing.

Figures on the first line means:
* There are 7095 documents,
* 5896 terms,
* and totally 247158 non-zero values within the document-term matrix.

The followin lines are the actual non-zero values, with the format of [Document No] [Term No] [Weight].

I hope my explanation make it clear for you. This is a common format, that’s why I have not seen a need for explanation.

]]>