neumoneumo / blog Goto Github PK

https://neumoneumo.github.io/blog/posts/existence-of-optimal-policy/

In this blog, we will prove the following theorem:
Optimal Policy Existence Theorem:
For any Markov Decision Process,
There exists an optimal policy $\pi_\star $ that is better than or equal to all other policies, $ \pi_\star \geq \pi , \forall \pi$ All optimal policies achieve the optimal value function, $V_{\pi_\star}(s)=V_\star(s)$ All optimal policies achieve the optimal action-value function $Q_{\pi_\star}(s,a)=Q_\star(s,a)$ Definition To simplify the exposition, we first define some basic concepts.

https://neumoneumo.github.io/blog/posts/va_list_problem/

What’s the output of the following codes? And Why?
#include <stdio.h>int main(int argc, char *argv[]) { printf("%#016llx\n", (char)0x80); printf("%#016llx\n", (unsigned char)0x80); return 0; } (You might encounter warnings informing you of the inconsistency between the specified format and the given arguments. Let’s neglect them.)
The answer is
0x00000000ffffff80 0x0000000000000080 Questions We have two questions:
Is it overloading that contributes to different behaviors when different types of arguments are passed. Why is the first output 0x00000000ffffff80 instead of 0xffffffffffffff80?

https://neumoneumo.github.io/blog/posts/build_docker/

In an docker container, you have full privileges to build the image of singularity or docker in it. But if only singularity is installed on the server and the root user sets up neither --fakeroot nor proot and you have exhausted your remote build minutes, what trick can you play to work around those restrictions?
Software Selection To solve the problem, we need an virtual machine under control on the server for enough privileges to execute singularity build(or docker build) which requires sudo if you meet such a tough condition as mentioned before.

https://neumoneumo.github.io/blog/posts/brainstorm/?

Here are some immature ideas that I come up with when I am taking a walk or having a meal. This post will be updated from time to time.
We have retrieval-augmented models which retrieve image or text to enhance the prediction accuracy. However, data in the form of image and text are redundant. Much of the information is not necessary. For instance, we do not need to remember every pixel, actually.

https://neumoneumo.github.io/blog/posts/qr_codes/

I have tried HARD to generate a photo of a brain or a neuron that can be scanned. But unfortunately, either it cannot be recognized as a QR code or it is irrelevant to my prompts.😭
It is much easier to generate that of girls or natural scenery though. Perhaps it’s better to choose another stable diffusion checkpoint. (By the way, thanks to Stable Diffusion WebUI, it is quite easy to deploy famous diffusion models.

https://neumoneumo.github.io/blog/posts/markov_property/

A stochastic process ${X_i}{i=0}^\infty $ is $n$-Markov if
$$P(X{t+n}|X_{t+n-1}, X_{t+n-2}, \cdots , X_{t}) = P(X_{t+n}|X_{t+n-1})$$
for any $t \ge 0$.
We would prove that
an $n$-Markov stochastic process must be $m$-Markov while is not necessarily $l$-Markov where $l &gt; n &gt; m$
N+1 to N First, we prove an (n+1)-Markov stochastic process must be n-Markov.
Proof: Suppose ${X_i}{i=0}^\infty$ is an $(n+1)$-Markov stochastic process. We have
$$P(X{t+n}|X_{t+n-1}, X_{t+n-2}, \cdots, X_t) = P(X_{t+n} | X_{t+n-1})$$ for any $t \ge 0$, deriving

https://neumoneumo.github.io/blog/posts/workflow_2023/

Main The conventional approach to storing a file involves fitting it into a hierarchical structure that necessitates a comprehensive overview of the corresponding field before the very first paper reading. You may place them flattened in an inbox folder before the tedious task of reindexing and categorizing hundreds of them hierarchically, otherwise, the overwhelming folder becomes your first obstacle to retrieve information. However, it can be deduced that both methods involve an additional burden of metal.