Giter Club home page Giter Club logo

loader's People

Contributors

kybxd avatar piratf avatar wenchy avatar youngerlee avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

loader's Issues

cpp: configuring column of `enum` type as `Index` lead to compiling errors

Compilation errors:

protoconf/cpp/equip_base_conf.pc.h:49:24:   required from here
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/hashtable_policy.h:1070:12: error: invalid use of incomplete type 'struct std::hash<protoconf::EquipQualityType>'
     struct _Hash_code_base<_Key, _Value, _ExtractKey, _H1, _H2,
            ^
In file included from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/basic_string.h:3033:0,
                 from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/string:52,
                 from common/extern/protobuf/src/google/protobuf/message.h:114,
                 from common/extern/protobuf/src/google/protobuf/util/json_util.h:36,
                 from protoconf/cpp/hub.pc.h:7,
                 from protoconf/cpp/registry.pc.h:7,
                 from protoconf/cpp/registry.pc.cc:6:
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/functional_hash.h:58:12: error: declaration of 'struct std::hash<protoconf::EquipQualityType>'
     struct hash;
            ^

My protoconf:

message EquipBaseConf {
  option (tableau.worksheet) = {name:"Sheet1" namerow:1 typerow:2 noterow:3 datarow:5 nameline:2 typeline:2 index:"SuitId" index:"Quality@QualityConf"};

  map<uint64, Equip> equip_map = 1 [(tableau.field) = {key:"EquipId" layout:LAYOUT_VERTICAL}];
  message Equip {
    uint64 equip_id = 1 [(tableau.field) = {name:"EquipId"}];
    string name = 2 [(tableau.field) = {name:"Name"}];
    protoconf.EquipWeaponPosType pos_type = 3 [(tableau.field) = {name:"PosType"}];
    uint32 role_id = 4 [(tableau.field) = {name:"RoleId"}];
    repeated int32 career_list = 5 [(tableau.field) = {name:"Career" layout:LAYOUT_INCELL}];
    int32 level = 6 [(tableau.field) = {name:"Level"}];
    protoconf.EquipQualityType quality = 7 [(tableau.field) = {name:"Quality"}];
    uint64 suit_id = 8 [(tableau.field) = {name:"SuitId"}];
    int32 sell_price = 9 [(tableau.field) = {name:"SellPrice"}];
    protoconf.BoolType can_upgrade = 10 [(tableau.field) = {name:"CanUpgrade"}];
    uint32 upgrade_id = 11 [(tableau.field) = {name:"UpgradeID"}];
    int32 upgrade_limit = 12 [(tableau.field) = {name:"UpgradeLimit"}];
    bool can_sold = 13 [(tableau.field) = {name:"CanSold"}];
    protoconf.ENMAvatarPos dress_type = 14 [(tableau.field) = {name:"DressType"}];
    uint32 resource_id = 15 [(tableau.field) = {name:"ResourceID"}];
    int32 upgrade_exp_provide = 16 [(tableau.field) = {name:"UpgradeExpProvide"}];
    int32 upgrade_gold_cost = 17 [(tableau.field) = {name:"UpgradeGoldCost"}];
    uint32 base_attr_id = 18 [(tableau.field) = {name:"BaseAttrId"}];
    uint32 star_up_id = 19 [(tableau.field) = {name:"StarUpId"}];
    repeated uint32 decompose_material_id_list = 20 [(tableau.field) = {name:"DecomposeMaterialId" layout:LAYOUT_INCELL}];
    repeated int32 decompose_material_num_list = 21 [(tableau.field) = {name:"DecomposeMaterialNum" layout:LAYOUT_INCELL}];
  }
}

Generated protoconf source code:

bool EquipBaseConf::ProcessAfterLoad() {
  // Index init.
  for (auto&& item1 : data_.equip_map()) {
    index_equip_map_[item1.second.suit_id()].push_back(&item1.second);
  }

  for (auto&& item1 : data_.equip_map()) {
    index_quality_conf_map_[item1.second.quality()].push_back(&item1.second);
  }

  return true;
}

go: support TreeMap

References

Implement treemap pkg in pathpkg/treemap

Define a new generic type Pair

To encapsule orderedmap and struct as a pair (as up-level orderedmap's value).

type Pair[T any, U any] struct {
    First T
    Second U
}

Generated APIs for ActivityConf

  • GetOrderedMap()
  • GetOrderedMap1(activityId uint64) (XXX, error)
  • GetOrderedMap2(activityId uint64, chapterId uint32) (XXX, error)
  • GetOrderedMap3(activityId uint64, chapterId uint32, sectionId uint32) (XXX, error)

go: check `nil` for getter APIs

  • Use generated protobuf-go getter APIs: GetXXX()
  • Support protobuf-go getter-like APIs which check nil

So user can call Get1 directly like: conf, err := hub.GetHub().GetActivityConf().Get1(id), and guarantee it will not panic.

type ActivityConf struct {
	data protoconf.ActivityConf
}

// Name returns the ActivityConf's message name.
func (x *ActivityConf) Name() string {
        if x == nil {
	        return ""
	}
	return string((&x.data).ProtoReflect().Descriptor().Name())
}

// Data returns the ActivityConf's inner message data.
func (x *ActivityConf) Data() *protoconf.ActivityConf {
        if x == nil {
                return nil
        }
	return &x.data
}

// Get1 finds value in the 1-level map. It will return nil if
// the deepest key is not found, otherwise return an error.
func (x *ActivityConf) Get1(activityId uint32) (*protoconf.ActivityConf_Activity, error) {
	d := x.Data().GetActivityMap()
	if d == nil {
		return nil, xerrors.Errorf(code.Nil, "ActivityMap is nil")
	}
	if val, ok := d[activityId]; !ok {
		return nil, xerrors.Errorf(code.NotFound, "activityId(%v)not found", activityId)
	} else {
		return val, nil
	}
}

Go: generated .*pc.go is wrong when use predefined message as map value

Reproduction

Predefined message Item in package base in file base.proto:

package base

message Item {
  uint32 id = 1 [(tableau.field).name = "ID"];
  int32 num = 2 [(tableau.field).name = "Num"];
}

Use predefined message Item as map value:

import "base.proto";

package protoconf

message ItemConf {
  option (tableau.worksheet) = {name:"ItemConf" namerow:1 typerow:2 noterow:3 datarow:4};

  map<uint32, base.Item> item_map = 1 [(tableau.field) = {key:"ID" layout:LAYOUT_VERTICAL}];
}

cpp(OrderedMap): includes middle-layer map value by tuple

GetOrderedMap only returns pair of std::map and ::google::protobuf::Map of the next layer, which may lead to the loss of middle-layer map value. Given such a protoconf:

message SkillResonanceV2Conf {
  option (tableau.worksheet) = {name:"Sheet1" namerow:1 typerow:2 noterow:3 datarow:5 nameline:2 typeline:2 ordered_map:true index:"RoleID"};

  map<uint32, Skill> skill_map = 1 [(tableau.field) = {key:"SkillID" layout:LAYOUT_VERTICAL}];
  message Skill {
    uint32 skill_id = 1 [(tableau.field) = {name:"SkillID"}];
    uint32 role_id = 2 [(tableau.field) = {name:"RoleID" prop:{refer:"RoleBaseConf.RoleID"}}];
    map<int32, Level> level_map = 3 [(tableau.field) = {key:"Level" layout:LAYOUT_VERTICAL}];
    message Level {
      int32 level = 1 [(tableau.field) = {name:"Level"}];
      protoconf.ENMTargetUnLock upgrade_condition_type = 2 [(tableau.field) = {name:"UpgradeConditionType"}];
      int64 upgrade_max_progress = 3 [(tableau.field) = {name:"UpgradeMaxProgress"}];
      repeated int64 upgrade_condition_param_list = 4 [(tableau.field) = {name:"UpgradeConditionParam" layout:LAYOUT_INCELL}];
      repeated uint32 upgrade_condition_array_param_list = 5 [(tableau.field) = {name:"UpgradeConditionArrayParam" layout:LAYOUT_INCELL}];
      map<uint32, int32> cost_material_map = 6 [(tableau.field) = {name:"CostMaterial" layout:LAYOUT_INCELL}];
      protoconf.AttrBuild resonance_addition = 7 [(tableau.field) = {name:"ResonanceAddition"}];
    }
  }
}

The generated C++ code is as follows. What confused me so much is that I cannot directly get protoconf::SkillResonanceV2Conf::Skill::Level through Skill_Level_OrderedMap.

  // OrderedMap accessers.
 public:
  using int32_OrderedMap = std::map<uint32_t, int32_t>;
  const int32_OrderedMap* GetOrderedMap(uint32_t skill_id, int32_t level) const;

  using int32_Map = ::google::protobuf::Map<uint32_t, int32_t>;
  using Skill_Level_OrderedMapValue = std::pair<int32_OrderedMap, const int32_Map*>;
  using Skill_Level_OrderedMap = std::map<int32_t, Skill_Level_OrderedMapValue>;
  const Skill_Level_OrderedMap* GetOrderedMap(uint32_t skill_id) const;

  using Skill_Level_Map = ::google::protobuf::Map<int32_t, protoconf::SkillResonanceV2Conf::Skill::Level>;
  using Skill_OrderedMapValue = std::pair<Skill_Level_OrderedMap, const Skill_Level_Map*>;
  using Skill_OrderedMap = std::map<uint32_t, Skill_OrderedMapValue>;
  const Skill_OrderedMap* GetOrderedMap() const;

 private:
  Skill_OrderedMap ordered_map_;

I hope that the middle-layer map value could be included into Skill_Level_OrderedMapValue. e.g.:

  // OrderedMap accessers.
 public:
  using int32_OrderedMap = std::map<uint32_t, int32_t>;
  const int32_OrderedMap* GetOrderedMap(uint32_t skill_id, int32_t level) const;

  using int32_Map = ::google::protobuf::Map<uint32_t, int32_t>;
  using Skill_Level_OrderedMapValue = std::tuple<int32_OrderedMap, const int32_Map*, protoconf::SkillResonanceV2Conf::Skill::Level*>;
  using Skill_Level_OrderedMap = std::map<int32_t,Skill_Level_OrderedMapValue >;
  const Skill_Level_OrderedMap* GetOrderedMap(uint32_t skill_id) const;

  using Skill_Level_Map = ::google::protobuf::Map<int32_t, protoconf::SkillResonanceV2Conf::Skill::Level>;
  using Skill_OrderedMapValue = std::tuple<Skill_Level_OrderedMap, const Skill_Level_Map*, protoconf::SkillResonanceV2Conf::Skill*>;
  using Skill_OrderedMap = std::map<uint32_t, Skill_OrderedMapValue>;
  const Skill_OrderedMap* GetOrderedMap() const;

 private:
  Skill_OrderedMap ordered_map_;

cpp: generate correct index accessors for incell-scalar-list field

Sheet:

企业微信截图_be6fd99c-e4e9-4e11-b0ef-abdb48fb8e8f

Generated:

message RecipeKeyMaterial {
  option (tableau.worksheet) = {name:"KeyMaterial" namerow:3 typerow:1 noterow:3 datarow:4 nameline:1 typeline:2 index:"KeyMaterials"};

  map<int32, RecipeKeyMaterialCfg> recipe_key_material_cfg_map = 1 [(tableau.field) = {key:"RecipeID" layout:LAYOUT_VERTICAL}];
  message RecipeKeyMaterialCfg {
    int32 recipe_id = 1 [(tableau.field) = {name:"RecipeID"}];
    repeated int32 key_materials_list = 2 [(tableau.field) = {name:"KeyMaterials" layout:LAYOUT_INCELL}];
  }
}

Error:
image

Go: add new method `Store` to store messager to file

Messager

type Messager interface {
Checker
// Name returns the unique message name.
Name() string
// Load fills inner message data from the specified direcotry and format.
Load(dir string, fmt format.Format, options ...load.Option) error
// ProcessAfterLoadAll is invoked after all messagers loaded.
ProcessAfterLoadAll(hub *Hub) error
}

Add method to interface Messager: Store() error

Hub

func (h *Hub) Load(dir string, filter Filter, format format.Format, options ...load.Option) error {

Add method to Hub: func (h *Hub) Store() error {

cpp: add hooks before hub switch

old hub -> load -> ProcessAfterLoad -> switch (L176) -> new hub

bool Hub::AsyncLoad(const std::string& dir, Filter filter /* = nullptr */, Format fmt /* = Format::kJSON */,
const LoadOptions* options /* = nullptr */) {
auto msger_container = LoadNewMessagerContainer(dir, filter, fmt, options);
if (!msger_container) {
return false;
}
sched_->Dispatch(std::bind(&Hub::SetMessagerContainer, this, msger_container));
return true;
}

  1. Add another messager-level hook ProcessAfterLoadAll which is called by hub-level hook PostProcess
  2. Add hub-level hook ProcessAfterLoad before switch (L176), this hook is for:
    • custom check
    • custom configuration aggregation
    • ...

load(cpp/go): support loading configs from multiple directories

CPP

Add an option:

// Paths maps each messager name to a corresponding config file path.
// If a messager name is existed, then the messager will be parsed from
// the config file directly.
// NOTE: only JSON, bin, and text formats are supported.
std::unordered_map<std::string, std::string> paths;

struct LoadOptions {
// Whether to ignore unknown JSON fields during parsing.
//
// https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.util.json_util#JsonParseOptions.
bool ignore_unknown_fields;
// postprocessor is called after loading all configurations.
Postprocessor postprocessor;
};

NOTE: set struct primitive field's default value: bool ignore_unknown_fields = false;

Go

TODO

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.