|
NAMETest2::Harness::Runner::Resource - Base class for resource management classesDESCRIPTIONSometimes you have limited resources that must be shared/divided between tests that run concurrently. Resource classes give you a way to leverage the IPC system used by Test2::Harness to manage resource assignment and recovery.SYNOPSISHere is a resource class that simply assigns an integer to each test. It would be possible to re-use integers, but since there are infinite integers this example is kept simple and just always grabs the next one.package Test2::Harness::Runner::Resource::Foo; use strict; use warnings; use parent 'Test2::Harness::Runner::Resource'; sub setup { my $class = shift; # NOT AN INSTANCE ... } sub available { my $self = shift; my ($task) = @_; # There are an infinite amount of integers, so we always return true return 1; } sub assign { my $self = shift; my ($task, $state) = @_; # Next ID, do not record the state change yet! my $id = 1 + ($self->{ID} //= 0); print "ASSIGN: $id = $task->{job_id}\n"; # 'record' should get whatever we need to record the resource, whatever you # pass in will become the argument to the record() sub below. This may be a # scalar, a hash, an array, etc. It will be serialized to JSON before # record() sees it. $state->{record} = $id; # Pass the resource into the test, this can be done as envronment variables # and/or arguments to the test (@ARGV). $state->{env_vars}->{FOO_ID} = $id; push @{$state->{args}} => $id; # The return is ignored. return; } sub record { my $self = shift; my ($job_id, $record_arg_from_assign) = @_; # The ID from $state->{record}->{$pkg} in assign. my $id = $record_arg_from_assign; # Update our internal state to reflect the new ID. $self->{ID} = $id; # Add a mapping of what job ID gets what integer ID. $self->{ID_TO_JOB_ID}->{$id} = $job_id; $self->{JOB_ID_TO_ID}->{$job_id} = $id; print "RECORD: $id = $job_id\n"; # The return is ignored } sub tick { my $self = shift; # This is called by only 1 process at a time and gives you a way to do # extra stuff at a regular interval without other processes trying to # do the same work at the same time. # For example, if a database is left in a dirty state after it is # released, you can fire off a cleanup action here knowing no other # process will run it at the same time. You can also be sure no record # messages will be sent while this sub is running as the process it # runs in has a lock. ... } sub release { my $self = shift; my ($job_id) = @_; # Clear the internal mapping, the integer ID is now free. Theoretically it # can be reused, but this example is not that complex. my $id = delete $self->{JOB_ID_TO_ID}->{$job_id}; # This is called for all tests that complete, even if they did not use # this resource, so we return if the job_id is not applicable. return unless defined $id; delete $self->{ID_TO_JOB_ID}->{$id}; print " FREE: $id = $job_id\n"; # The return is ignored } sub cleanup { my $self = shift; print "CLEANUP!\n"; } 1; The print statements generated will look like this when running 2 tests concurrently: yath test -R Foo -j2 t/testA.t t/testB.t [...] (INTERNAL) ASSIGN: 1 = 4F7CF5F6-E43F-11EA-9199-24FCBF610F44 (INTERNAL) RECORD: 1 = 4F7CF5F6-E43F-11EA-9199-24FCBF610F44 (INTERNAL) ASSIGN: 2 = E19CD98C-E436-11EA-8469-8DF0BF610F44 (INTERNAL) RECORD: 2 = E19CD98C-E436-11EA-8469-8DF0BF610F44 (INTERNAL) FREE: 1 = 4F7CF5F6-E43F-11EA-9199-24FCBF610F44 (INTERNAL) FREE: 2 = E19CD98C-E436-11EA-8469-8DF0BF610F44 (INTERNAL) CLEANUP! [...] Depending on the tests run the 'FREE' prints may be out of order. WORKFLOWHOW STATE IS MANAGEDDepending on your preload configuration, yath may have several runners launching tests. If a runner has nothing to do it will lock the queue and try to find the next test that should be run. Only 1 of the runners will be in control of the queue at any given time, but the control of the queue may pass between runners. To manage this there is a mechanism to record messages that allow each runner to maintain a copy of the current state.CHECK IF RESOURCES ARE AVAILABLEEach runner will have an instance of your resource class. When the runner is in control of the queue, and wants to designate the next test to run, it will check with the resource classes to make sure the correct resources are available. To do that it will call "available($task)" on each resource instance.The $task will contain the specification for the test, it is a hashref, and you SHOULD NOT modify it. The only key most people care about is the 'file' key, which has the test file that will be run if resources are available. If resources are available, or if the specific file does not need the resource, the "available()" method should return true. If the file does need your resource(s), and none are available, this should return false. If any resource class returns false it means the test cannot be run yet and the runner will look for another test to run. ASSIGN A RESOURCEIf the runner has determined the test can be run, and all necessary resources are available, it will then call "assign($task, $state)" on all resource class instances. At this time the resource class should decide what resource(s) to assign to the class.CRITICAL NOTE: the "assing()" method MUST NOT alter any internal state on the resource class instance. State modification must wait for the "record()" method to be called. This is because the "assign()" method is only called in one runner process, the "record()" method call will happen in every runner process to insure they all have the same internal state. The assign() sub should modify the $state hash, which has 3 keys:
RECORD A RESOURCEOnce a resource is assigned, a message will be sent to all runner processes INCLUDING THE ONE THAT DID THE ASSIGN that says it should call "record($job_id, $record_val)" on your resource class instance. Your resource class instance must use this to update the state so that once done ALL processes will have the proper internal state.The $record_val is whatever you put into "$state->{record}" in the "assign()" method above. QUEUE MANAGEMENT IS UNLOCKEDOnce the above has been done, queue management will be unlocked. You can be guarenteed that only one process will be run the "available()", and "assign()" sequence at a time, and that they will be called in order, though "assign()" may not be called if another resource was not available. If "assign()" is called, you can be guarenteed that all processes, including the one that called "assign()" will have their "record()" called with the proper argument BEFORE they try to manage the queue (which is the only place resources are checked or assigned).RELEASE A RESOURCEWhenever a process that is using a resource exits, the runner that waits on that process will eventually send an IPC message announcing that the job_id has completed. Every time a job_id completes the "release($job_id)" method will be called on your resource class in all runner processes. This allows the state to be updated to reflect the freed resource.You can be guarenteed that any process that locks the queue to run a new test will eventually see the message. The message may come in during a loop that is checking for resources, in which case the state will not reflect the resource being available, however in such cases the loop will end and be called again later with the message having been receieved. There will be no deadlock due to a queue manager waiting for the message. There are no guarentees about what order resources will be released in. METHODS
SOURCEThe source code repository for Test2-Harness can be found at http://github.com/Test-More/Test2-Harness/.MAINTAINERS
AUTHORS
COPYRIGHTCopyright 2020 Chad Granum <exodist7@gmail.com>.This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See http://dev.perl.org/licenses/
Visit the GSP FreeBSD Man Page Interface. |