SDM'Studio

Class sdm::SolvableByHSVI

Class List > sdm > SolvableByHSVI

Public interface for problems that can be solved using HSVI (i.e. beliefMDP, occupancyMDP, occupancyGame, etc).HSVI can be used to solve the problem that implement this interface.

  • #include <solvable_by_hsvi.hpp>

Inherited by the following classes: sdm::SolvableByMDP

Public Functions

Type Name
virtual double do_excess (double incumbent, double lb_value, double ub_value, double cost_so_far, double error, number t) = 0
Compute the excess of the HSVI paper. It refers to the termination condition.
virtual std::shared_ptr< Space > getActionSpaceAt (const std::shared_ptr< State > & state, number t) = 0
Get the action space at a specific state and timestep. The state dependency is required when the game forbid the usage of a number of actions in this state. It is also used in some reformulated problems where actions are decision rules. The time dependency is required in extensive-form games in which some agents have a different action space.
virtual double getDiscount (number t) const = 0
Get the specific discount factor for the problem at hand.
virtual double getExpectedNextValue (const std::shared_ptr< ValueFunction > & value_function, const std::shared_ptr< State > & state, const std::shared_ptr< Action > & action, number t) = 0
Get the expected next value.
virtual std::shared_ptr< State > getInitialState () = 0
Get the initial state.
virtual Pair< std::shared_ptr< State >, double > getNextState (const std::shared_ptr< ValueFunction > & value_function, const std::shared_ptr< State > & belief, const std::shared_ptr< Action > & action, const std::shared_ptr< Observation > & observation, number t) = 0
virtual std::shared_ptr< Space > getObservationSpaceAt (const std::shared_ptr< State > & state, const std::shared_ptr< Action > & action, number t) = 0
virtual double getReward (const std::shared_ptr< State > & state, const std::shared_ptr< Action > & action, number t) = 0
Get the reward of executing a specific action in an specific state at timestep t. The time dependency can be required in non-stationnary problems.
virtual const std::shared_ptr< MDPInterface > & getUnderlyingProblem () const = 0
Get the well defined underlying problem. Some problems are solvable by DP algorithms even if they are not well defined. Usually, they simply are reformulation of an underlying well defined problem. For instance, the underlying DecPOMDP of the OccupancyMDP or the underlying POMDP of the current BeliefMDP.
virtual double getWeightedDiscount (number t) = 0
Get the specific weighted discount factor for the problem at hand.
virtual bool isSerialized () const = 0
Check if the problem is serialized.
virtual std::shared_ptr< State > nextState (const std::shared_ptr< State > & state, const std::shared_ptr< Action > & action, number t=0, const std::shared_ptr< HSVI > & hsvi=nullptr) = 0
Select the next state.
virtual Pair< std::shared_ptr< Action >, double > selectNextAction (const std::shared_ptr< ValueFunction > & lb, const std::shared_ptr< ValueFunction > & ub, const std::shared_ptr< State > & state, number t) = 0
Select the next action.
virtual void setInitialState (const std::shared_ptr< State > &) = 0
virtual ~SolvableByHSVI ()

Public Functions Documentation

function do_excess

virtual double sdm::SolvableByHSVI::do_excess (
    double incumbent,
    double lb_value,
    double ub_value,
    double cost_so_far,
    double error,
    number t
) = 0

Parameters:

  • double : incumbent
  • double : lb value
  • double : ub value
  • double : cost_so_far
  • double : error
  • number : horizon

Returns:

double

function getActionSpaceAt

virtual std::shared_ptr< Space > sdm::SolvableByHSVI::getActionSpaceAt (
    const std::shared_ptr< State > & state,
    number t
) = 0

Parameters:

  • state the state
  • t the timestep

Returns:

the action space

function getDiscount

virtual double sdm::SolvableByHSVI::getDiscount (
    number t
) const = 0

Parameters:

  • number decision epoch or any other parameter

Returns:

double discount factor

function getExpectedNextValue

virtual double sdm::SolvableByHSVI::getExpectedNextValue (
    const std::shared_ptr< ValueFunction > & value_function,
    const std::shared_ptr< State > & state,
    const std::shared_ptr< Action > & action,
    number t
) = 0

Parameters:

  • value_function a pointer on the value function to use to perform the calculus.
  • state the state on which to evaluate the next expected value.
  • action
  • t

Returns:

the expected next value

function getInitialState

virtual std::shared_ptr< State > sdm::SolvableByHSVI::getInitialState () = 0

function getNextState

virtual Pair < std::shared_ptr< State >, double > sdm::SolvableByHSVI::getNextState (
    const std::shared_ptr< ValueFunction > & value_function,
    const std::shared_ptr< State > & belief,
    const std::shared_ptr< Action > & action,
    const std::shared_ptr< Observation > & observation,
    number t
) = 0

function getObservationSpaceAt

virtual std::shared_ptr< Space > sdm::SolvableByHSVI::getObservationSpaceAt (
    const std::shared_ptr< State > & state,
    const std::shared_ptr< Action > & action,
    number t
) = 0

function getReward

virtual double sdm::SolvableByHSVI::getReward (
    const std::shared_ptr< State > & state,
    const std::shared_ptr< Action > & action,
    number t
) = 0

Parameters:

  • state the state
  • action the action
  • t the timestep

Returns:

the reward

function getUnderlyingProblem

virtual const std::shared_ptr< MDPInterface > & sdm::SolvableByHSVI::getUnderlyingProblem () const = 0

Returns:

the underlying problem

function getWeightedDiscount

virtual double sdm::SolvableByHSVI::getWeightedDiscount (
    number t
) = 0

Parameters:

  • number decision epoch or any other parameter

Returns:

double discount factor

function isSerialized

virtual bool sdm::SolvableByHSVI::isSerialized () const = 0

Returns:

true if the problem is serialized.

Returns:

false if the problem is not serialized.

function nextState

virtual std::shared_ptr< State > sdm::SolvableByHSVI::nextState (
    const std::shared_ptr< State > & state,
    const std::shared_ptr< Action > & action,
    number t=0,
    const std::shared_ptr< HSVI > & hsvi=nullptr
) = 0

Parameters:

  • state the state
  • action the action
  • t the timestep
  • hsvi a pointer on the algorithm that makes the call

Returns:

the next state

function selectNextAction

virtual Pair < std::shared_ptr< Action >, double > sdm::SolvableByHSVI::selectNextAction (
    const std::shared_ptr< ValueFunction > & lb,
    const std::shared_ptr< ValueFunction > & ub,
    const std::shared_ptr< State > & state,
    number t
) = 0

Parameters:

  • lb the lower bound
  • ub the upper bound
  • state the current state
  • t the timestep

Returns:

the selected action

function setInitialState

virtual void sdm::SolvableByHSVI::setInitialState (
    const std::shared_ptr< State > &
) = 0

function ~SolvableByHSVI

inline virtual sdm::SolvableByHSVI::~SolvableByHSVI ()

The documentation for this class was generated from the following file src/sdm/world/solvable_by_hsvi.hpp