test_DoctoredBuffer.c File Reference

Test cases for Doctored buffer. More...

#include <stdbool.h>
#include "nomos.h"
#include "doctorBuffer_utils.h"
#include <stdio.h>
#include <stdlib.h>
#include <CUnit/CUnit.h>
#include <stdarg.h>
#include "nomos_utils.h"
#include "util.h"
#include "list.h"
#include "licenses.h"
#include "process.h"
#include "nomos_regex.h"
#include "_autodefs.h"

Include dependency graph for test_DoctoredBuffer.c:

Go to the source code of this file.

Functions
void	test_doctorBuffer ()
	Test for doctorBuffer() More...
void	test_doctorBuffer_uncollapse ()
	Test for uncollapsePosition() More...
static void	report_Match (char *buf)
	Helper function to match licenses and highlight info.
void	test_doctorBuffer_fromFile ()
	Test for idxGrep_recordPosition() More...
void	test_1_removeHtmlComments ()
	Test for removeHtmlComments() More...
void	test_2_removeLineComments ()
	Test for removeLineComments() More...
void	test_3_cleanUpPostscript ()
	Test for cleanUpPostscript() More...
void	test_4_removeBackslashesAndGTroffIndicators ()
	Test for removeBackslashesAndGTroffIndicators() More...
void	test_5_convertWhitespaceToSpaceAndRemoveSpecialChars ()
	Test for convertWhitespaceToSpaceAndRemoveSpecialChars() More...
void	test_6_dehyphen ()
	Test for dehyphen() More...
void	test_6a_dehyphen ()
	Test for dehyphen() More...
void	test_7_removePunctuation ()
	Test for removePunctuation() More...
void	test_8_ignoreFunctionCalls ()
	Test for ignoreFunctionCalls() More...
void	test_9_convertSpaceToInvisible ()
	Test for convertSpaceToInvisible() More...
void	test_10_compressDoctoredBuffer ()
	Test for compressDoctoredBuffer() More...

Variables
CU_TestInfo	doctorBuffer_testcases []

Detailed Description

Test cases for Doctored buffer.

Definition in file test_DoctoredBuffer.c.

Function Documentation

void test_10_compressDoctoredBuffer

(

)

Test for compressDoctoredBuffer()

Test:

1. garbage collect: eliminate all INVISIBLE characters in the buffer

Definition at line 459 of file test_DoctoredBuffer.c.

void test_1_removeHtmlComments

(

)

Test for removeHtmlComments()

The test are in the order of the function calls in doctorBuffer While the test result does not depend on the order. If possible the output string of the first test is the input string of the second one and so on. Clear exceptions: Input cannot be MarkUp and PostScript simultaneously.. Please do not change the test order in this source file

Test:

Step 1: take care of embedded HTML/XML and special HTML-chars like " and   – but DON'T remove the text in an HTML comment. There might be licensing text/information in the comment!

Later on (in parseLicenses()) we search for URLs in the raw-text

Definition at line 203 of file test_DoctoredBuffer.c.

void test_2_removeLineComments

(

)

Test for removeLineComments()

Test:

Step 2: remove comments that start at the beginning of a line, * like ^dnl, ^xcomm, ^comment, and //

Definition at line 226 of file test_DoctoredBuffer.c.

void test_3_cleanUpPostscript

(

)

Test for cleanUpPostscript()

Test:

Step 3 - strip out crap at end-of-line on postscript documents

Definition at line 250 of file test_DoctoredBuffer.c.

void test_4_removeBackslashesAndGTroffIndicators

(

)

Test for removeBackslashesAndGTroffIndicators()

Test:

Step 4: remove groff/troff font-size indicators, the literal string backslash-n and all backslahes, ala: ==> `perl -pe 's,\s[+-][0-9]*,,g;s,\s[0-9]*,,g;s/\n//g;' |f`

Definition at line 271 of file test_DoctoredBuffer.c.

void test_5_convertWhitespaceToSpaceAndRemoveSpecialChars

(

)

Test for convertWhitespaceToSpaceAndRemoveSpecialChars()

Test:

Step 5: convert white-space to real spaces, and remove unnecessary punctuation, ala: ==> `tr -d '*=+#$|%.,:;!?()\][\140\047\042' | tr '\011\012\015' ' '`

Note

we purposely do NOT process backspace-characters here. Perhaps there's an improvement in the wings for this?

Definition at line 300 of file test_DoctoredBuffer.c.

void test_6_dehyphen

(

)

Test for dehyphen()

Look for hyphenations of words, to compress both halves into a sin- gle (sic) word. Regex == "[a-z]- [a-z]".

Test:

1. Create dirty string with hyphens
2. Call dehyphen() on the dirty string
3. Check if the string is cleaned

Note

not sure this will work based on the way we strip punctuation out of the buffer above – work on this later.

Definition at line 337 of file test_DoctoredBuffer.c.

void test_6a_dehyphen

(

)

Test for dehyphen()

Test:

1. Create a string with mix of hyphens and INVISIBLE
2. Call dehyphen() and check the result

Definition at line 355 of file test_DoctoredBuffer.c.

void test_7_removePunctuation

(

)

Test for removePunctuation()

Test:

Step 6: clean up miscellaneous punctuation, ala: ==> perl -pe 's,[-_/]+ , ,g;s/print[_a-zA-Z]* //g;s/ / /g;'

Note the added ',' so removePunctuation does ignore commas...

Definition at line 380 of file test_DoctoredBuffer.c.

void test_8_ignoreFunctionCalls

(

)

Test for ignoreFunctionCalls()

Test:

Ignore function calls to print routines: only concentrate on what's being printed (sometimes programs do print licensing information) – but don't ignore real words that END in 'print', like footprint and fingerprint. Here, we take a risk and just look for a 't' (in "footprint"), or for an 'r' (in "fingerprint"). If someone has ever coded a print routine that is named 'rprint' or tprint', we're spoofed.

Definition at line 409 of file test_DoctoredBuffer.c.

void test_9_convertSpaceToInvisible

(

)

Test for convertSpaceToInvisible()

Test:

1. Create string with many empty spaces
2. Call convertSpaceToInvisible()
3. Check if all spaces (>2) are replaced with INVISIBLE

Definition at line 434 of file test_DoctoredBuffer.c.

void test_doctorBuffer

(

)

Test for doctorBuffer()

Test:

1. Create a dirty string
2. Initialize scanner and call doctorBuffer() on dirty string
3. Check if the string is sanitized

Definition at line 49 of file test_DoctoredBuffer.c.

void test_doctorBuffer_fromFile

(

)

Test for idxGrep_recordPosition()

Test:

1. Load a license info from a file
2. Add license to the current matches
3. Call idxGrep_recordPosition() to match license regex
4. Call report_Match() to verify the match

Definition at line 128 of file test_DoctoredBuffer.c.

void test_doctorBuffer_uncollapse

(

)

Test for uncollapsePosition()

Test:

1. Create a dirty string
2. Initialize scanner and call doctorBuffer() on dirty string
3. Check if uncollapsePosition() refers to original position

Definition at line 75 of file test_DoctoredBuffer.c.

Variable Documentation

CU_TestInfo doctorBuffer_testcases[]

Initial value:

=
{
{ "Testing doctorBuffer:", test_doctorBuffer },
{ "Testing doctorBufer_uncollapse", test_doctorBuffer_uncollapse },
{ "Testing doctorBuffer_fromFile", test_doctorBuffer_fromFile },
{ "Testing removeHtmlComents:", test_1_removeHtmlComments },
{ "Testing removeLineComments:", test_2_removeLineComments },
{ "Testing cleanUpPostscript:", test_3_cleanUpPostscript },
{ "Testing removeBackslashes:", test_4_removeBackslashesAndGTroffIndicators },
{ "Testing convertWhitespace:", test_5_convertWhitespaceToSpaceAndRemoveSpecialChars },
{ "Testing dehyphen:", test_6_dehyphen },
{ "Testing dehyphen2:", test_6a_dehyphen },
{ "Testing removePunctuation:", test_7_removePunctuation },
{ "Testing ignoreFunctionCalls:", test_8_ignoreFunctionCalls },
{ "Testing convertSpaceToInvisible:", test_9_convertSpaceToInvisible },
{ "Testing compressDoctoredBuffer:", test_10_compressDoctoredBuffer },
CU_TEST_INFO_NULL }

Definition at line 479 of file test_DoctoredBuffer.c.