Changeset 2317


Ignore:
Timestamp:
Feb 11, 2010 11:33:46 AM (10 years ago)
Author:
wehart
Message:

Merged revisions 2246-2316 via svnmerge from
​https://software.sandia.gov/svn/public/coopr/coopr.pysp/trunk

........

r2246 | wehart | 2010-02-01 21:10:48 -0700 (Mon, 01 Feb 2010) | 2 lines


Tagging coopr.pysp release 2.2

........

r2247 | wehart | 2010-02-01 21:46:08 -0700 (Mon, 01 Feb 2010) | 2 lines


Documentation fix.

........

r2248 | jwatson | 2010-02-03 08:44:59 -0700 (Wed, 03 Feb 2010) | 3 lines


Adding os.path.expanduser wrappers around all directory/filenames, to facilitate correct processing of ~ characters.

........

r2249 | jwatson | 2010-02-03 09:28:25 -0700 (Wed, 03 Feb 2010) | 3 lines


Misc fixes.

........

r2254 | jwatson | 2010-02-03 21:09:38 -0700 (Wed, 03 Feb 2010) | 3 lines


Changed PyomoModelData? call from add_data_mumble() to add().

........

r2255 | jwatson | 2010-02-03 21:38:35 -0700 (Wed, 03 Feb 2010) | 3 lines


Re-factoring of PH options parser code to accomodate MRP work being done by DLW.

........

r2256 | jwatson | 2010-02-03 22:18:11 -0700 (Wed, 03 Feb 2010) | 3 lines


Major speed improvements in the EF writer by avoiding Python deep-copes - saves a few orders of magnitude of run-time.

........

r2260 | jwatson | 2010-02-04 21:44:29 -0700 (Thu, 04 Feb 2010) | 3 lines


Refactoring of ph initialization routines to support sampling and bundling.

........

r2261 | jwatson | 2010-02-04 21:46:37 -0700 (Thu, 04 Feb 2010) | 3 lines


Renaming ph_script module to phinit, which is more accurate with the newly factored, library-like functionality.

........

r2262 | jwatson | 2010-02-04 21:54:50 -0700 (Thu, 04 Feb 2010) | 3 lines


Minor improvement to phinit functionality.

........

r2267 | jwatson | 2010-02-05 15:09:02 -0700 (Fri, 05 Feb 2010) | 3 lines


Initial PySP unit tests!!!

........

r2268 | wehart | 2010-02-05 15:25:52 -0700 (Fri, 05 Feb 2010) | 2 lines


A fix to the tests.

........

r2269 | jwatson | 2010-02-05 15:46:35 -0700 (Fri, 05 Feb 2010) | 3 lines


Added SIZES3 PySP test and added absolute paths to "runph" script.

........

r2270 | jwatson | 2010-02-05 16:01:08 -0700 (Fri, 05 Feb 2010) | 3 lines


PySP tests need absolute output paths!

........

r2271 | jwatson | 2010-02-05 16:20:01 -0700 (Fri, 05 Feb 2010) | 3 lines


Making unit tests for PySP compatible with coverage utilities. Farmer examples work, SIZES3 doesn't for some reason.

........

r2272 | jwatson | 2010-02-05 16:59:46 -0700 (Fri, 05 Feb 2010) | 3 lines


Various fixes to PySP unit tests. Changing name of testphextension to examplephextension - with the test prefix, coverage tests import the module, which causes all kinds of issues.

........

r2275 | jwatson | 2010-02-06 13:45:11 -0700 (Sat, 06 Feb 2010) | 3 lines


Testing improvements. From lpython, the tests run individually just fine. In aggregate, only the first run passes - independent of what test that might be! Something to stare at later....

........

r2277 | jwatson | 2010-02-06 23:30:13 -0700 (Sat, 06 Feb 2010) | 3 lines


Fixed problem with runph --profile option, broken by my recent factoring of phinit.py.

........

r2288 | jwatson | 2010-02-08 13:32:41 -0700 (Mon, 08 Feb 2010) | 3 lines


Significant initialization speed reductions in the WW PH extension for PySP.

........

r2290 | jwatson | 2010-02-08 19:22:32 -0700 (Mon, 08 Feb 2010) | 1 line


Initial commit of multi-stage capacity expansion problem in PySP

........

r2291 | jwatson | 2010-02-08 19:23:24 -0700 (Mon, 08 Feb 2010) | 1 line


Miscellaneous fix to ef writer involving indexed cost variables

........

r2296 | jwatson | 2010-02-09 18:56:48 -0700 (Tue, 09 Feb 2010) | 3 lines


Removing monster-sized LP file from the PySP forestry examples directory.

........

r2297 | jwatson | 2010-02-09 18:59:05 -0700 (Tue, 09 Feb 2010) | 3 lines


The extensive forms in the PySP forestry example were massive - and are now gone.

........

r2298 | jwatson | 2010-02-09 19:03:09 -0700 (Tue, 09 Feb 2010) | 1 line


Removing output logs for PySP network flow example

........

r2299 | jwatson | 2010-02-09 19:04:12 -0700 (Tue, 09 Feb 2010) | 1 line


Removing a big network flow EF

........

r2300 | jwatson | 2010-02-09 19:05:08 -0700 (Tue, 09 Feb 2010) | 3 lines


Removing PySP cap example EF to free up space.

........

r2301 | jwatson | 2010-02-09 19:14:26 -0700 (Tue, 09 Feb 2010) | 1 line


Performance improvements to PH obtained by processing scenario sub-problem results as they come in, instead of waiting for them after a solver barrier sync

........

Location:
coopr.pysp/stable/2.3
Files:
23 deleted
9 edited
436 copied

Legend:

Unmodified
Added
Removed

  • coopr.pysp/stable/2.3

  • coopr.pysp/stable/2.3/README.txt

    r1806 r2317  
    1 ================
     1=================
    22coopr.pysp README
    3 ================
     3=================
    44
    55This Python package defines stochastic programming extensions for the Pyomo

  • coopr.pysp/stable/2.3/coopr/pysp/__init__.py

    r2201 r2317  
    2020from ef import *
    2121from ef_writer_script import *
    22 from ph_script import *
     22from phinit import *
    2323
    2424pyutilib.component.core.PluginGlobals.pop_env()

  • coopr.pysp/stable/2.3/coopr/pysp/ef.py

    r2110 r2317  
    186186               node_data_filename = scenario_data_directory_name + os.sep + current_node._name + ".dat"
    187187#               print "Node data for scenario=" + scenario._name + " partially loading from file=" + node_data_filename
    188                scenario_data.add_data_file(node_data_filename)
     188               scenario_data.add(node_data_filename)
    189189               current_node = current_node._parent
    190190            scenario_data.read(model=scenario_instance)
    … …  
    222222      for (stage_variable, index_template, stage_variable_indices) in stage._variables:
    223223
    224          print "Creating master variable and blending constraints for decision variable=", stage_variable, ", indices=", stage_variable_indices
     224         print "Creating master variable and blending constraints for decision variable=", stage_variable, ", indices=",stage_variable_indices
    225225
    226226         for tree_node in stage._tree_nodes:
    … …  
    237237                  master_variable = getattr(master_binding_instance, master_variable_name)
    238238               except:
    239                   # the deepcopy is probably too expensive (and unnecessary) computationally -
    240                   # easier to just use the constructor with the stage variable index/bounds/etc.
    241                   # NOTE: need to re-assign the master variables for each _varval - they probably
    242                   #       point to a bogus model.
    243                   new_master_variable = copy.deepcopy(stage_variable)
    244                   new_master_variable.name = master_variable_name
     239                  new_master_variable_index = stage_variable._index
     240                  new_master_variable = None
     241                  if (len(new_master_variable_index) is 1) and (None in new_master_variable_index):
     242                     new_master_variable = Var(name=stage_variable.name)
     243                  else:
     244                     new_master_variable = Var(new_master_variable_index, name=stage_variable.name)
     245                  new_master_variable.construct()
    245246                  new_master_variable._model = master_binding_instance
    246247                  setattr(master_binding_instance, master_variable_name, new_master_variable)
     248
     249                  # TBD - TECHNICALLY, WE NEED TO COPY BOUNDS - BUT WE REALLY DON'T, AS THEY ARE ON THE PER-INSTNACE VARS!
    247250
    248251                  master_variable = new_master_variable
    … …  
    313316      for tree_node in stage._tree_nodes:
    314317
     318         # TBD - the following is bad - check to see if it's already there (I suspect some of them are!!!)         
     319
     320         # this is undoubtedly wasteful, in that a cost variable
     321         # for each tree node is created with *all* indices.         
    315322         new_cost_variable_name = tree_node._name + "_" + cost_variable.name
    316 
    317          # TBD - the following is bad - check to see if it's already there (I suspect some of them are!!!)
    318 
    319          # this is undoubtedly wasteful, in that a cost variable
    320          # for each tree node is created with *all* indices.
    321          new_cost_variable = copy.deepcopy(cost_variable)
    322          new_cost_variable.name = new_cost_variable_name
     323         new_cost_variable_index = cost_variable._index
     324         new_cost_variable = None
     325         if (len(new_cost_variable_index) is 1) and (None in new_cost_variable_index):
     326            new_cost_variable = Var(name=new_cost_variable_name)
     327         else:
     328            new_cost_variable = Var(new_cost_variable_index, name=new_cost_variable_name)
     329         new_cost_variable.construct()
    323330         new_cost_variable._model = master_binding_instance
    324          setattr(master_binding_instance, new_cost_variable_name, new_cost_variable)
     331         setattr(master_binding_instance, new_cost_variable_name, new_cost_variable)         
    325332
    326333         # the following is necessary, specifically to get the name - deepcopy won't reset these attributes.
    … …  
    644651                  master_variable = getattr(master_binding_instance, master_variable_name)
    645652               except:
    646                   # the deepcopy is probably too expensive (and unnecessary) computationally -
    647                   # easier to just use the constructor with the stage variable index/bounds/etc.
    648                   # NOTE: need to re-assign the master variables for each _varval - they probably
    649                   #       point to a bogus model.
    650                   new_master_variable = copy.deepcopy(stage_variable)
    651                   new_master_variable.name = master_variable_name
     653                  new_master_variable_index = stage_variable._index
     654                  new_master_variable = None
     655                  if (len(new_master_variable_index) is 1) and (None in new_master_variable_index):
     656                     new_master_variable = Var(name=stage_variable.name)
     657                  else:
     658                     new_master_variable = Var(new_master_variable_index, name=stage_variable.name)
     659                  new_master_variable.construct()
    652660                  new_master_variable._model = master_binding_instance
    653661                  setattr(master_binding_instance, master_variable_name, new_master_variable)
     662
     663                  # TBD - TECHNICALLY, WE NEED TO COPY BOUNDS - BUT WE REALLY DON'T, AS THEY ARE ON THE PER-INSTNACE VARS!
    654664
    655665                  master_variable = new_master_variable
    … …  
    711721         # this is undoubtedly wasteful, in that a cost variable
    712722         # for each tree node is created with *all* indices.
    713          new_cost_variable = copy.deepcopy(cost_variable)
    714          new_cost_variable.name = new_cost_variable_name
     723         new_cost_variable_name = tree_node._name + "_" + cost_variable.name
     724         new_cost_variable_index = cost_variable._index
     725         new_cost_variable = None
     726         if (len(new_cost_variable_index) is 1) and (None in new_cost_variable_index):
     727            new_cost_variable = Var(name=new_cost_variable_name)
     728         else:
     729            new_cost_variable = Var(new_cost_variable_index, new_cost_variable_name)
     730         new_cost_variable.construct()
    715731         new_cost_variable._model = master_binding_instance
    716          setattr(master_binding_instance, new_cost_variable_name, new_cost_variable)
     732         setattr(master_binding_instance, new_cost_variable_name, new_cost_variable)                 
    717733
    718734         # the following is necessary, specifically to get the name - deepcopy won't reset these attributes.

  • coopr.pysp/stable/2.3/coopr/pysp/ef_writer_script.py

    r1979 r2317  
    8484      risk_alpha = options.risk_alpha
    8585
    86    write_ef_from_scratch(options.model_directory, options.instance_directory, options.output_file, \
     86   write_ef_from_scratch(os.path.expanduser(options.model_directory), os.path.expanduser(options.instance_directory), os.path.expanduser(options.output_file), \
    8787                         generate_weighted_cvar, cvar_weight, risk_alpha)
    8888

  • coopr.pysp/stable/2.3/coopr/pysp/ph.py

    r2201 r2317  
    851851                  if self._verbose is True:
    852852                     print "Node data for scenario=" + scenario._name + " partially loading from file=" + node_data_filename
    853                   scenario_data.add_data_file(node_data_filename)
     853                  scenario_data.add(node_data_filename)
    854854                  current_node = current_node._parent
    855855               scenario_data.read(model=scenario_instance)
    … …  
    10091009
    10101010      action_handles = []
    1011       action_handle_instance_map = {}
     1011      scenario_action_handle_map = {} # maps scenario names to action handles
     1012      action_handle_scenario_map = {} # maps action handles to scenario names
    10121013
    10131014      for scenario in self._scenario_tree._scenarios:
    … …  
    10301031
    10311032         new_action_handle = self._solver_manager.queue(instance, opt=self._solver, tee=self._output_solver_log)
    1032          action_handle_instance_map[scenario._name] = new_action_handle
     1033         scenario_action_handle_map[scenario._name] = new_action_handle
     1034         action_handle_scenario_map[new_action_handle] = scenario._name
    10331035
    10341036         action_handles.append(new_action_handle)
    10351037
    1036       # STEP 2: barrier sync for all scenario sub-problem solves.
     1038      # STEP 2: loop for the solver results, reading them and loading
     1039      #         them into instances as they are available.
     1040
    10371041      if self._verbose is True:
    10381042         print "Waiting for scenario sub-problem solves"
    1039       self._solver_manager.wait_all(action_handles)
    1040       if self._verbose is True:     
    1041          print "Scenario sub-problem solves completed"     
    1042 
    1043       solve_end_time = time.time()
    1044       self._cumulative_solve_time += (solve_end_time - solve_start_time)
    1045 
    1046       if self._output_times is True:
    1047          print "Aggregate sub-problem solve time=%8.2f" % (solve_end_time - solve_start_time)
    1048 
    1049       # STEP 3: Load the results!
    1050       for scenario_name, action_handle in action_handle_instance_map.items():
    1051 
    1052          if self._verbose is True:         
    1053             print "Successfully processed results for scenario="+scenario_name
    1054 
    1055          instance = self._instances[scenario_name]
    1056          results = self._solver_manager.get_results(action_handle)
     1043
     1044      num_results_so_far = 0
     1045
     1046      while (num_results_so_far < len(self._scenario_tree._scenarios)):
     1047
     1048         action_handle = self._solver_manager.wait_any()
     1049         results = self._solver_manager.get_results(action_handle)         
     1050         scenario_name = action_handle_scenario_map[action_handle]
     1051         instance = self._instances[scenario_name]         
     1052
     1053         if self._verbose is True:
     1054            print "Results obtained for scenario="+scenario_name
    10571055
    10581056         if len(results.solution) == 0:
    … …  
    10621060         if self._output_solver_results is True:
    10631061            print "Results for scenario=",scenario_name
    1064             results.write(num=1)           
     1062            results.write(num=1)
    10651063
    10661064         start_time = time.time()
    … …  
    10681066         end_time = time.time()
    10691067         if self._output_times is True:
    1070             print "Time loading results="+str(end_time-start_time)+" seconds"
     1068            print "Time loading results into instance="+str(end_time-start_time)+" seconds"
    10711069
    10721070         if self._verbose is True:                 
    10731071            print "Successfully loaded solution for scenario="+scenario_name
     1072
     1073         num_results_so_far = num_results_so_far + 1
     1074         
     1075      if self._verbose is True:     
     1076         print "Scenario sub-problem solves completed"     
     1077
     1078      solve_end_time = time.time()
     1079      self._cumulative_solve_time += (solve_end_time - solve_start_time)
     1080
     1081      if self._output_times is True:
     1082         print "Aggregate sub-problem solve time this iteration=%8.2f" % (solve_end_time - solve_start_time)
    10741083
    10751084      if self._verbose is True:
    … …  
    13521361   def iteration_k_solve(self):
    13531362
    1354      if self._verbose is True:
    1355         print "------------------------------------------------"       
    1356         print "Starting PH iteration " + str(self._current_iteration) + " solves"
    1357 
    1358      # cache the objective values generated by PH for output at the end of this function.
    1359      ph_objective_values = {}
    1360 
    1361      solve_start_time = time.time()
    1362 
    1363      # STEP 0: set up all global solver options.
    1364      self._solver.mipgap = self._mipgap     
    1365 
    1366      # STEP 1: queue up the solves for all scenario sub-problems and
    1367      #         grab all of the action handles for the subsequent barrier sync.
    1368 
    1369      action_handles = []
    1370      action_handle_instance_map = {}
    1371 
    1372      for scenario in self._scenario_tree._scenarios:     
    1373 
    1374         instance = self._instances[scenario._name]
    1375 
    1376         if self._verbose is True:
    1377            print "Queuing solve for scenario=" + scenario._name
    1378 
    1379         # IMPT: You have to re-presolve, as the simple presolver collects the linear terms together. If you
    1380         # don't do this, you won't see any chance in the output files as you vary the problem parameters!
    1381         # ditto for instance fixing!
    1382         instance.presolve()
    1383 
    1384         # once past iteration 0, there is always a feasible solution from which to warm-start.
    1385         # however, you might want to disable warm-start when the solver is behaving badly (which does happen).
    1386         new_action_handle = None
    1387         if (self._disable_warmstarts is False) and (self._solver.warm_start_capable() is True):
    1388            new_action_handle = self._solver_manager.queue(instance, opt=self._solver, warmstart=True, tee=self._output_solver_log)
    1389         else:
    1390            new_action_handle = self._solver_manager.queue(instance, opt=self._solver, tee=self._output_solver_log)           
    1391 
    1392         action_handle_instance_map[scenario._name] = new_action_handle
    1393 
    1394         action_handles.append(new_action_handle)
    1395 
    1396      # STEP 2: barrier sync for all scenario sub-problem solves.
    1397      if self._verbose is True:           
    1398         print "Waiting for scenario sub-problem solves"
    1399      self._solver_manager.wait_all(action_handles)
    1400      if self._verbose is True:               
    1401         print "Scenario sub-problem solves completed"
    1402        
    1403      solve_end_time = time.time()
    1404      self._cumulative_solve_time += (solve_end_time - solve_start_time)
    1405 
    1406      if self._output_times is True:
    1407         print "Aggregate sub-problem solve time=%8.2f" % (solve_end_time - solve_start_time)
    1408 
    1409      # STEP 3: Load the results!
    1410      for scenario_name, action_handle in action_handle_instance_map.items():
    1411 
    1412         if self._verbose is True:         
    1413            print "Successfully processed results for scenario="+scenario_name
    1414 
    1415         instance = self._instances[scenario_name]
    1416         results = self._solver_manager.get_results(action_handle)
    1417 
    1418         if len(results.solution) == 0:
    1419            results.write()
    1420            raise RuntimeError, "Solve failed for scenario="+scenario_name+"; no solutions generated"
    1421 
    1422         if self._output_solver_results is True:
    1423            print "Results for scenario=", scenario_name
    1424            results.write(num=1)           
    1425 
    1426         start_time = time.time()
    1427         instance.load(results)
    1428         end_time = time.time()
    1429 
    1430         if self._output_times is True:
    1431            print "Time loading results="+str(end_time-start_time)+" seconds"       
    1432 
    1433         if self._verbose is True:                 
    1434            print "Successfully loaded solution for scenario="+scenario_name
    1435 
    1436         # we're assuming there is a single solution.
    1437         # the "value" attribute is a pre-defined feature of any solution - it is relative to whatever
    1438         # objective was selected during optimization, which of course should be the PH objective.
    1439         ph_objective_values[instance.name] = float(results.solution(0).objective['f'].value)
    1440 
    1441      if self._verbose is True:
    1442         print "Successfully completed PH iteration " + str(self._current_iteration) + " solves - solution statistics:"
    1443         print "  Scenario             PH Objective             Cost Objective"
    1444         for scenario in self._scenario_tree._scenarios:
    1445            instance = self._instances[scenario._name]
    1446            for objective_name in instance.active_components(Objective):
    1447               objective = instance.active_components(Objective)[objective_name]
    1448               print "%20s       %18.4f     %14.4f" % (scenario._name, ph_objective_values[scenario._name], 0.0)
     1363      if self._verbose is True:
     1364         print "------------------------------------------------"       
     1365         print "Starting PH iteration " + str(self._current_iteration) + " solves"
     1366
     1367      # cache the objective values generated by PH for output at the end of this function.
     1368      ph_objective_values = {}
     1369
     1370      solve_start_time = time.time()
     1371
     1372      # STEP 0: set up all global solver options.
     1373      self._solver.mipgap = self._mipgap     
     1374
     1375      # STEP 1: queue up the solves for all scenario sub-problems and
     1376      #         grab all of the action handles for the subsequent barrier sync.
     1377
     1378      action_handles = []
     1379      scenario_action_handle_map = {} # maps scenario names to action handles
     1380      action_handle_scenario_map = {} # maps action handles to scenario names
     1381
     1382      for scenario in self._scenario_tree._scenarios:     
     1383
     1384         instance = self._instances[scenario._name]
     1385
     1386         if self._verbose is True:
     1387            print "Queuing solve for scenario=" + scenario._name
     1388
     1389         # IMPT: You have to re-presolve, as the simple presolver collects the linear terms together. If you
     1390         # don't do this, you won't see any chance in the output files as you vary the problem parameters!
     1391         # ditto for instance fixing!
     1392         instance.presolve()
     1393
     1394         # once past iteration 0, there is always a feasible solution from which to warm-start.
     1395         # however, you might want to disable warm-start when the solver is behaving badly (which does happen).
     1396         new_action_handle = None
     1397         if (self._disable_warmstarts is False) and (self._solver.warm_start_capable() is True):
     1398            new_action_handle = self._solver_manager.queue(instance, opt=self._solver, warmstart=True, tee=self._output_solver_log)
     1399         else:
     1400            new_action_handle = self._solver_manager.queue(instance, opt=self._solver, tee=self._output_solver_log)           
     1401
     1402         scenario_action_handle_map[scenario._name] = new_action_handle
     1403         action_handle_scenario_map[new_action_handle] = scenario._name         
     1404
     1405         action_handles.append(new_action_handle)
     1406
     1407      # STEP 2: loop for the solver results, reading them and loading
     1408      #         them into instances as they are available.
     1409      if self._verbose is True:
     1410         print "Waiting for scenario sub-problem solves"
     1411
     1412      num_results_so_far = 0
     1413
     1414      while (num_results_so_far < len(self._scenario_tree._scenarios)):
     1415
     1416         action_handle = self._solver_manager.wait_any()
     1417         results = self._solver_manager.get_results(action_handle)         
     1418         scenario_name = action_handle_scenario_map[action_handle]
     1419         instance = self._instances[scenario_name]         
     1420
     1421         if self._verbose is True:
     1422            print "Results obtained for scenario="+scenario_name
     1423
     1424         if len(results.solution) == 0:
     1425            results.write(num=1)
     1426            raise RuntimeError, "Solve failed for scenario="+scenario_name+"; no solutions generated"
     1427
     1428         if self._output_solver_results is True:
     1429            print "Results for scenario=",scenario_name
     1430            results.write(num=1)
     1431
     1432         start_time = time.time()
     1433         instance.load(results)
     1434         end_time = time.time()
     1435         if self._output_times is True:
     1436            print "Time loading results into instance="+str(end_time-start_time)+" seconds"
     1437
     1438         if self._verbose is True:                 
     1439            print "Successfully loaded solution for scenario="+scenario_name
     1440
     1441         # we're assuming there is a single solution.
     1442         # the "value" attribute is a pre-defined feature of any solution - it is relative to whatever
     1443         # objective was selected during optimization, which of course should be the PH objective.
     1444         ph_objective_values[instance.name] = float(results.solution(0).objective['f'].value)
     1445
     1446         num_results_so_far = num_results_so_far + 1
     1447         
     1448      if self._verbose is True:     
     1449         print "Scenario sub-problem solves completed"     
     1450
     1451      solve_end_time = time.time()
     1452      self._cumulative_solve_time += (solve_end_time - solve_start_time)
     1453
     1454      if self._output_times is True:
     1455         print "Aggregate sub-problem solve time this iteration=%8.2f" % (solve_end_time - solve_start_time)
     1456
     1457      if self._verbose is True:
     1458         print "Successfully completed PH iteration " + str(self._current_iteration) + " solves - solution statistics:"
     1459         print "  Scenario             PH Objective             Cost Objective"
     1460         for scenario in self._scenario_tree._scenarios:
     1461            instance = self._instances[scenario._name]
     1462            for objective_name in instance.active_components(Objective):
     1463               objective = instance.active_components(Objective)[objective_name]
     1464               print "%20s       %18.4f     %14.4f" % (scenario._name, ph_objective_values[scenario._name], 0.0)
    14491465
    14501466   def solve(self):

  • coopr.pysp/stable/2.3/coopr/pysp/tests/unit/test_ph.py

    r1768 r2317  
    55import os
    66import sys
     7import string
    78from os.path import abspath, dirname
    89sys.path.insert(0, dirname(dirname(abspath(__file__)))+"/../..")
    9 currdir = dirname(abspath(__file__))+os.sep
     10
     11current_directory = dirname(abspath(__file__))+os.sep
     12
     13pysp_examples_dir = dirname(dirname(dirname(dirname(dirname(abspath(__file__))))))+os.sep+"examples"+os.sep+"pysp"+os.sep
     14
     15coopr_bin_dir = dirname(dirname(dirname(dirname(dirname(dirname(dirname(abspath(__file__))))))))+os.sep+"bin"+os.sep
     16
    1017#
    1118# Import the testing packages
    … …  
    1421import pyutilib.misc
    1522import pyutilib.th
     23import pyutilib.subprocess
     24import coopr.pysp
     25import coopr.plugins.mip
     26import coopr.pysp.phinit
     27
     28def filter_time(line):
     29   return "seconds" in line
     30
     31cplex = None
     32cplex_available = False
     33try:
     34    cplex = coopr.plugins.mip.CPLEX(keepFiles=True)
     35    cplex_available = (not cplex.executable() is None) and cplex.available(False)
     36except pyutilib.common.ApplicationError:
     37    cplex_available=False
     38
     39glpk = None
     40glpk_available = False
     41try:
     42    glpk = coopr.plugins.mip.GLPK(keepFiles=True)
     43    glpk_available = (not glpk.executable() is None) and glpk.available(False)
     44except pyutilib.common.ApplicationError:
     45    glpk_available=False
    1646
    1747#
    … …  
    2252class TestPH(pyutilib.th.TestCase):
    2353
    24     #
    25     # A test which redirects IO to a file, prints stuff, resets the
    26     # redirection, and then compares with a baseline file.
    27     #
    28     def test_dummy(self):
    29         pyutilib.misc.setup_redirect(currdir+"dummy.out")
    30         print "HELLO WORLD"
     54    def cleanup(self):
     55
     56        # IMPT: This step is key, as Python keys off the name of the module, not the location.
     57        #       So, different reference models in different directories won't be detected.
     58        #       If you don't do this, the symptom is a model that doesn't have the attributes
     59        #       that the data file expects.
     60        if "ReferenceModel" in sys.modules:
     61           del sys.modules["ReferenceModel"]
     62
     63    def test_quadratic_farmer(self):
     64
     65        if cplex_available is False:
     66           return
     67
     68        pyutilib.misc.setup_redirect(current_directory+"farmer_quadratic.out")
     69        farmer_examples_dir = pysp_examples_dir + "farmer"
     70        model_dir = farmer_examples_dir + os.sep + "models"
     71        instance_dir = farmer_examples_dir + os.sep + "scenariodata"       
     72        argstring = "runph --solver=cplex --solver-manager=serial --model-directory="+model_dir+" --instance-directory="+instance_dir
     73        args = string.split(argstring)
     74        coopr.pysp.phinit.run(args=args)
    3175        pyutilib.misc.reset_redirect()
    32         self.failUnlessFileEqualsBaseline(currdir+"dummy.out",currdir+"dummy.txt")
     76        self.cleanup()       
     77        self.failUnlessFileEqualsBaseline(current_directory+"farmer_quadratic.out",current_directory+"farmer_quadratic.baseline", filter=filter_time)
     78       
     79    def test_linearized_farmer(self):
     80
     81        if cplex_available is False:
     82           return       
     83
     84        pyutilib.misc.setup_redirect(current_directory+"farmer_linearized.out")
     85        farmer_examples_dir = pysp_examples_dir + "farmer"
     86        model_dir = farmer_examples_dir + os.sep + "models"
     87        instance_dir = farmer_examples_dir + os.sep + "scenariodata"       
     88        argstring = "runph --solver=cplex --solver-manager=serial --model-directory="+model_dir+" --instance-directory="+instance_dir+" --linearize-nonbinary-penalty-terms=10"
     89        args = string.split(argstring)
     90        coopr.pysp.phinit.run(args=args)       
     91        pyutilib.misc.reset_redirect()
     92        self.cleanup()               
     93        self.failUnlessFileEqualsBaseline(current_directory+"farmer_linearized.out",current_directory+"farmer_linearized.baseline", filter=filter_time)
     94
     95    def test_quadratic_sizes3(self):
     96
     97        if cplex_available is False:
     98           return
     99       
     100        pyutilib.misc.setup_redirect(current_directory+"sizes3_quadratic.out")
     101
     102        sizes_example_dir = pysp_examples_dir + "sizes"
     103        model_dir = sizes_example_dir + os.sep + "models"
     104        instance_dir = sizes_example_dir + os.sep + "SIZES3"       
     105        argstring = "runph --solver=cplex --solver-manager=serial --model-directory="+model_dir+" --instance-directory="+instance_dir+ \
     106                    " --max-iterations=200"+ \
     107                    " --rho-cfgfile="+sizes_example_dir+os.sep+"config"+os.sep+"rhosetter.cfg"+ \
     108                    " --scenario-solver-options=mip_tolerances_integrality=1e-7"+ \
     109                    " --enable-ww-extensions"+ \
     110                    " --ww-extension-cfgfile="+sizes_example_dir+os.sep+"config"+os.sep+"wwph.cfg"+ \
     111                    " --ww-extension-suffixfile="+sizes_example_dir+os.sep+"config"+os.sep+"wwph.suffixes"
     112        args = string.split(argstring)
     113        coopr.pysp.phinit.run(args=args)               
     114        pyutilib.misc.reset_redirect()
     115        self.failUnlessFileEqualsBaseline(current_directory+"sizes3_quadratic.out",current_directory+"sizes3_quadratic.baseline", filter=filter_time)
     116        self.cleanup()       
     117
     118    def run(self, result=None):
     119       global cplex_available
     120       if not cplex_available:
     121          return
     122       pyutilib.th.TestCase.run(self,result)
    33123
    34124if __name__ == "__main__":

  • coopr.pysp/stable/2.3/coopr/pysp/wwphextension.py

    r2201 r2317  
    2929   return cmp(value_b, value_a)
    3030
    31 #==================================================
    3231#==================================================
    3332class wwphextension(SingletonPlugin):
    … …  
    266265               tree_node._fixed_var_flag = {}
    267266
    268                # next, create parameters for each variable in the corresponding tree node.
    269 
    270267               for (variable, index_template, variable_indices) in stage._variables:
    271268
    272269                  variable_name = variable.name
    273270                  variable_type = variable.domain
     271
     272                  # create the parameters for all indices, even though this might be a bit
     273                  # wasteful. in practice, this probably won't be the case, and determining
     274                  # the reduced index set would probably take longer that it is worth.
     275
     276                  new_stat_index = variable._index
     277                  new_stat_parameter_name = "NODESTAT_NUM_ITERS_CONVERGED_"+variable.name
     278                  new_stat_parameter = None
     279                  # this bit of ugliness is due to Pyomo not correctly handling the Param construction
     280                  # case when the supplied index set consists strictly of None, i.e., the source variable
     281                  # is a singleton. this case be cleaned up when the source issue in Pyomo is fixed.                     
     282                  if (len(new_stat_index) is 1) and (None in new_stat_index):
     283                     new_stat_parameter = Param(name=new_stat_parameter_name)
     284                  else:
     285                     new_stat_parameter = Param(new_stat_index,name=new_stat_parameter_name)
     286                  for newindex in new_stat_index:
     287                     new_stat_parameter[newindex] = 0
     288                  tree_node._num_iters_converged[variable.name] = new_stat_parameter
     289                     
     290                  # need to know to what we have most recently converged
     291                  new_conv_index = variable._index
     292                  new_conv_parameter_name = "NODESTAT_LAST_CONVERGED_VAL_"+variable.name
     293                  new_conv_parameter = None
     294                  if (len(new_conv_index) is 1) and (None in new_conv_index):
     295                     new_conv_parameter = Param(name=new_conv_parameter_name)
     296                  else:
     297                     new_conv_parameter = Param(new_conv_index,name=new_conv_parameter_name)
     298                  for newindex in new_conv_index:
     299                     new_conv_parameter[newindex] = 0.5 # not an int, so harmless
     300                  tree_node._last_converged_val[variable.name] = new_conv_parameter
     301                     
     302                  # need to know to what has been fixed
     303                  new_fix_index = variable._index
     304                  new_fix_parameter_name = "NODESTAT_FIXED_FLAG_VAL_"+variable.name
     305                  new_fix_parameter = None
     306                  if (len(new_fix_index) is 1) and (None in new_fix_index):
     307                     new_fix_parameter = Param(name=new_fix_parameter_name)
     308                  else:
     309                     new_fix_parameter = Param(new_fix_index,name=new_fix_parameter_name)
     310                  for newindex in new_fix_index:
     311                     new_fix_parameter[newindex] = False
     312                  tree_node._fixed_var_flag[variable.name] = new_fix_parameter
     313
     314                  # now make the w hash value storage array
     315                  new_hash_index = variable._index
     316                  new_hash_parameter_name = "W_HASH_STORAGE_"+variable.name
     317                  new_hash_parameter = None
     318                  if (len(new_hash_index) is 1) and (None in new_hash_index):
     319                     new_hash_parameter = Param(ph._iteration_index_set, name=new_hash_parameter_name)
     320                  else:
     321                     new_hash_parameter = Param(new_hash_index, ph._iteration_index_set, name=new_hash_parameter_name)
     322                  for newindex in new_hash_index:
     323                     for i in range(0, ph._max_iterations+1):
     324                        # the following if-then block is a complete hack, due to the
     325                        # fact that we can't index by None if the Param is unary.
     326                        if new_hash_parameter.dim() == 1:
     327                           new_hash_parameter[i] = 0
     328                        else:
     329                           new_hash_parameter[newindex,i] = 0
     330                  tree_node._w_hash[variable.name] = new_hash_parameter
    274331                 
     332                  # JPW has no idea why the following code block is here, or if it is necessary.
    275333                  for index in variable_indices:
    276334
    … …  
    278336                     #       variable values in the last stage of a stochastic program will *not*
    279337                     #       have a defined _stage attribute.
    280 #                     print "dlw debug create stage association ",variable_name, index, stage._name                     
    281338                     variable[index]._stage = stage
    282339
    283                      new_stat_index = variable._index
    284                      new_stat_parameter_name = "NODESTAT_NUM_ITERS_CONVERGED_"+variable.name
    285                      new_stat_parameter = None
    286                      # this bit of ugliness is due to Pyomo not correctly handling the Param construction
    287                      # case when the supplied index set consists strictly of None, i.e., the source variable
    288                      # is a singleton. this case be cleaned up when the source issue in Pyomo is fixed.                     
    289                      if (len(new_stat_index) is 1) and (None in new_stat_index):
    290                         new_stat_parameter = Param(name=new_stat_parameter_name)
    291                      else:
    292                         new_stat_parameter = Param(new_stat_index,name=new_stat_parameter_name)
    293                      for newindex in new_stat_index:
    294                         new_stat_parameter[newindex] = 0
    295                      tree_node._num_iters_converged[variable.name] = new_stat_parameter
    296                      
    297                      # need to know to what we have most recently converged
    298                      new_conv_index = variable._index
    299                      new_conv_parameter_name = "NODESTAT_LAST_CONVERGED_VAL_"+variable.name
    300                      new_conv_parameter = None
    301                      if (len(new_conv_index) is 1) and (None in new_conv_index):
    302                         new_conv_parameter = Param(name=new_conv_parameter_name)
    303                      else:
    304                         new_conv_parameter = Param(new_conv_index,name=new_conv_parameter_name)
    305                      for newindex in new_conv_index:
    306                         new_conv_parameter[newindex] = 0.5 # not an int, so harmless
    307                      tree_node._last_converged_val[variable.name] = new_conv_parameter
    308                      
    309                      # need to know to what has been fixed
    310                      new_fix_index = variable._index
    311                      new_fix_parameter_name = "NODESTAT_FIXED_FLAG_VAL_"+variable.name
    312                      new_fix_parameter = None
    313                      if (len(new_fix_index) is 1) and (None in new_fix_index):
    314                         new_fix_parameter = Param(name=new_fix_parameter_name)
    315                      else:
    316                         new_fix_parameter = Param(new_fix_index,name=new_fix_parameter_name)
    317                      for newindex in new_fix_index:
    318                         new_fix_parameter[newindex] = False
    319                      tree_node._fixed_var_flag[variable.name] = new_fix_parameter
    320                      
    321                      # now make the w hash value storage array
    322                      new_hash_index = variable._index
    323                      new_hash_parameter_name = "W_HASH_STORAGE_"+variable.name
    324                      new_hash_parameter = None
    325                      if (len(new_hash_index) is 1) and (None in new_hash_index):
    326                         new_hash_parameter = Param(ph._iteration_index_set, name=new_hash_parameter_name)
    327                      else:
    328                         new_hash_parameter = Param(new_hash_index, ph._iteration_index_set, name=new_hash_parameter_name)
    329                      for newindex in new_hash_index:
    330                         for i in range(0, ph._max_iterations+1):
    331                            # the following if-then block is a complete hack, due to the
    332                            # fact that we can't index by None if the Param is unary.
    333                            if new_hash_parameter.dim() == 1:
    334                               new_hash_parameter[i] = 0
    335                            else:
    336                               new_hash_parameter[newindex,i] = 0
    337                      tree_node._w_hash[variable.name] = new_hash_parameter
    338340
    339341      # store the total variable counts for future reporting.

  • coopr.pysp/stable/2.3/scripts/runph

    r1979 r2317  
    4646    curr = os.path.dirname(curr)
    4747
    48 import coopr.pysp.ph_script
     48import coopr.pysp.phinit
    4949
    5050try:
    51     coopr.pysp.ph_script.run()
     51    coopr.pysp.phinit.run()
    5252except ValueError, str:
    5353    print "VALUE ERROR:"
Note: See TracChangeset for help on using the changeset viewer.