How to create a KOTH CP that its timer counts up?

You can change the count direction of a team_round_timer in its keyvalues.

Only one tiny issue here. You are not the one who creates the timers. That's automatic. You can't set the keyvalues of what you don't create.

Or can you? If you know the not very secret entity names, you can use AddOutput to set new keyvalues.

Yes, that's something I didn't realize. If you're using KOTH logic, the timers are created automatically at runtime and are called zz_red_koth_timer and zz_blue_koth_timer respectively. I'm not sure what would happen if you add these timers manually. Best case scenario, everything works as intended; worst case scenario, it breaks the game logic and, in the most unlikely scenario, makes your map crash on launch. If you can't place the timers manually, you can change the keyvalues of any entity (including the KOTH timers) using the addoutput input.

You should be able to create a logic_auto (or another entity, just with different output names) with outputs set to these to cause your KOTH timers to count upwards. Keep in mind that you'll need to start/stop the timers by triggering zz_red/blue_koth_timer with the inputs Resume or Pause, and if you want to have the timer send an output at a specific time you'd probably be best creating a math_counter for each timer that increase by one every second (by having a logic_timer with a refire interval of one sending an output OnTimer to your counter inputting Add with a parameter override of one) that the timers are active, and either sending an output OnHitMax, or, if you want multiple outputs, sending an output OutValue to a logic_case without setting a parameter override