[oi-dev] COMPONENT_TEST_TRANSFORMS for PARI

[stes@PANDORA.BE]

http://docs.openindiana.org/dev/userland/

requires an update to first define "test" as a TARGET

because test is not listed in the table after 

"To build a component you simply cd into the directory of the software, and type "gmake TARGET" (here we use gmake to call GNU make), where TARGET can be one of:"

some targets are listed but "test" is not one of them.

Further down there is just one paragraph as documentation about the whole process:

"It's advised to put the expected test ouput in test/results-BITS.master (where BITS are either 32 or 64) and to ensure that the gmake test target generates reproducible results. You can use the COMPONENT_TEST_TRANSFORMS variable to set a list of sed directives to transform the test output and make it reproducible."

This is not entirely correct as BITS can also be "all" instead of 32 or 64.

Also from your reply I understand that there are 2 usages:

  1) run "gmake test" to run a set of tests
  2) run "gmake test" to run a test-and-compare if test/results-BITS.master already exists

So basically "gmake sample-test" something like that could create an empty tests/results-BITS.master,
just like "sample-manifest" is creating a sample manifest.

----- Op 14 okt 2022 om 1:53 schreef Tim Mooney via oi-dev oi-dev at openindiana.org:

> In regard to: [oi-dev] COMPONENT_TEST_TRANSFORMS for PARI, stes at PANDORA.BE...:
> 
>> Because PARI/gp is using its own Configure do I have to define
>> COMPONENT_TEST_TRANSFORMS to remove all compile/build output ??
> 
> Probably.  Whenever I wanted to have comparable test results for
> a component, I found it easiest to use a filter to delete everything
> up to the start of the tests, and then additional filters to remove
> anything that would be system-specific or variable within the tests
> (e.g. timing for test runs, dates and times, system name, etc.).
> 
>> I am not sure the documentation on oi-userland build is sufficient
>> regarding to "gmake test".
> 
> Especially for someone just starting out, the current documentation
> alone is probably not enough.  You're not a beginner, but even you
> have run into questions.
> 
> When I was starting out, I got good help from Alexander, Michal, Aurelien,
> etc.  Most of the time the help was pointing me at an existing example in
> a different component, that did something similar to what I needed to do
> for the component I was stuck on.
> 
> What might be useful is for you to inventory the components that have
> test output for comparison (under various possible names), and then check
> the Makefile for these to find what COMPONENT_TEST_TRANSFORMS are in
> place.  That should give you a lot of examples to start with.  I know that
> for the perl components I worked on and any libraries, I tried to
> make certain there was a comparison file for the test suite.
> 
>> Any suggestions ?
> 
> If I was updating a component that didn't have a test comparison file yet,
> I would start by creating an empty one, via something like
> 
> 	touch test/results-all.master
> 
> (or a BITS-specific variant or alternate output name).
> 
> I would generally then start running the test suite, tweaking the
> COMPONENT_TEST_TRANSFORMS to filter out more of the output, and manually
> updating the starting comparison file.  Frequently this involved saving
> "diff" output and adding it to the comparison file while stripping the
> diff markers like "+" at the beginning of a line and line-number markers.
> 
> As long as you're familiar with the "diff" format, it's pretty easy, and
> if you have a bunch of components to update the repetition will get you
> to the point where you're very comfortable with the process.
> 
> Tim
> --
> Tim Mooney                                             Tim.Mooney at ndsu.edu
> Enterprise Computing & Infrastructure /
> Division of Information Technology    /                701-231-1076 (Voice)
> North Dakota State University, Fargo, ND 58105-5164
> 
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev