MDPtoolbox

  The MDP toolbox proposes functions related to the resolution of discrete-time Markov Decision Process : finite horizon, value iteration, policy iteration, linear programming algorithms with some variants. The functions (m-functions) were developped with MATLAB v6.0 (one of the functions requires the Mathworks Optimization Toolbox) by the decision team of the Biometry and Artificial Intelligence Unit of INRA Toulouse (France). The version 2.0 (February 2005) handles sparse matrices and contains an example

MDPtoolbox
..........\documentation
..........\.............\arrow.gif
..........\.............\DOCUMENTATION.html
..........\.............\index_alphabetic.html
..........\.............\index_category.html


..........\.............\mdp_bellman_operator.html
..........\.............\mdp_bellman_operator_policy.html
..........\.............\mdp_check.html
..........\.............\mdp_check_square_stochastic.html
..........\.............\mdp_computePR.html
..........\.............\mdp_eval_policy.html
..........\.............\mdp_finite_horizon.html
..........\.............\mdp_LP.html
..........\.............\mdp_policy_iteration.html
..........\.............\mdp_policy_iteration_modified.html
..........\.............\mdp_rand.html
..........\.............\mdp_relative_value_iteration.html
..........\.............\mdp_span.html
..........\.............\mdp_value_iteration.html
..........\.............\mdp_value_iterationGS.html
..........\.............\mdp_value_iteration_bound_iter.html
..........\.............\mdp_verbose_silent.html
..........\examples
..........\........\race
..........\........\....\action_to_acceleration.m
..........\........\....\compute_transitions.m
..........\........\....\convert_values_to_state.m
..........\........\....\data
..........\........\....\....\100x100
..........\........\....\....\.......\100_100_track.txt
..........\........\....\....\10x10
..........\........\....\....\.....\10_10_track.txt
..........\........\....\....\.....\trans_matrix_-1_-1.dat
..........\........\....\....\.....\trans_matrix_-1_0.dat
..........\........\....\....\.....\trans_matrix_-1_1.dat
..........\........\....\....\.....\trans_matrix_0_-1.dat
..........\........\....\....\.....\trans_matrix_0_0.dat
..........\........\....\....\.....\trans_matrix_0_1.dat
..........\........\....\....\.....\trans_matrix_1_-1.dat
..........\........\....\....\.....\trans_matrix_1_0.dat
..........\........\....\....\.....\trans_matrix_1_1.dat
..........\........\....\....\11x11
..........\........\....\....\.....\11_11_track.txt
..........\........\....\....\25x25
..........\........\....\....\.....\25_25_track.txt
..........\........\....\....\.....\trans_matrix_-1_-1.dat
..........\........\....\....\.....\trans_matrix_-1_0.dat
..........\........\....\....\.....\trans_matrix_-1_1.dat
..........\........\....\....\.....\trans_matrix_0_-1.dat
..........\........\....\....\.....\trans_matrix_0_0.dat
..........\........\....\....\.....\trans_matrix_0_1.dat
..........\........\....\....\.....\trans_matrix_1_-1.dat
..........\........\....\....\.....\trans_matrix_1_0.dat
..........\........\....\....\.....\trans_matrix_1_1.dat
..........\........\....\....\4x4
..........\........\....\....\...\4_4_track.txt
..........\........\....\....\50x50
..........\........\....\....\.....\50_50_track.txt
..........\........\....\....\7x7
..........\........\....\....\...\7_7_track.txt
..........\........\....\display_race.m
..........\........\....\documentation
..........\........\....\.............\circuit.ps
..........\........\....\.............\circuit2.ps
..........\........\....\.............\elig-cumul.eps
..........\........\....\.............\elig-replace.eps
..........\........\....\.............\french2.sty
..........\........\....\.............\race_documentation.pdf
..........\........\....\.............\race_documentation.tex
..........\........\....\.............\sparse2.ps
..........\........\....\.............\successeurs.ps
..........\........\....\.............\transfo.ps
..........\........\....\generate_starting_state.m
..........\........\....\gen_transition_matrix.m
..........\........\....\get_values_from_state.m
..........\........\....\index_computation.m
..........\........\....\main.m
..........\........\....\race_end.m
..........\........\....\read_data.m
..........\........\....\safe_transition.m
..........\........\....\simulate_transitions.m
..........\........\....\trajectory.m
..........\........\....\transition_matrix.m
..........\mdp_bellman_operator.m
..........\mdp_bellman_operator_policy.m
..........\mdp_check.m
..........\mdp_check_square_stochastic.m
..........\mdp_computePR.m
..........\mdp_eval_policy.m
..........\mdp_finite_horizon.m
..........\mdp_LP.m
..........\mdp_policy_iteration.m
..........\mdp_policy_iteration_modified.m
..........\mdp_rand.m
..........\mdp_relative_value_iteration.m
..........\mdp_silent.m
..........\mdp_span.m