Quentin/doluedu-fastapi-template
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor & reorder some points (#2)

Browse Source 
* Add clarifications for sqlalchemy, migrations, db points

* Add better explanation for background tasks

* Reorder points and break links
This commit is contained in:
Yerassyl
2022-08-26 01:27:51 +06:00
committed by GitHub
parent 9ec5226456
commit 308a7253e2
1 changed files with 57 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

95
README.md
View File

@@ -18,21 +18,19 @@ Some of them are worth sharing.
10. [Use Pydantic's BaseSettings for configs](https://github.com/zhanymkanov/fastapi-best-practices#10-use-pydantics-basesettings-for-configs)
11. [SQLAlchemy: Set DB keys naming convention from day 0](https://github.com/zhanymkanov/fastapi-best-practices#11-sqlalchemy-set-db-keys-naming-convention-from-day-0)
12. [Set DB table naming convention immediately from day 0](https://github.com/zhanymkanov/fastapi-best-practices#12-set-db-table-naming-convention-immediately-from-day-0)
13. ~~Set UUIDs within the app~~
14. [Set tests client async from day 0](https://github.com/zhanymkanov/fastapi-best-practices#14-set-tests-client-async-from-day-0)
15. ~~Set postgres identity from day 0~~
16. [Use BackgroundTasks](https://github.com/zhanymkanov/fastapi-best-practices#16-use-backgroundtasks)
17. [Typing is important](https://github.com/zhanymkanov/fastapi-best-practices#17-typing-is-important)
18. [Don't hope your clients will send small BLOBs. Save files in chunk](https://github.com/zhanymkanov/fastapi-best-practices#18-dont-hope-your-clients-will-send-small-blobs-save-files-in-chunk)
19. [Be careful with dynamic pydantic fields](https://github.com/zhanymkanov/fastapi-best-practices#19-be-careful-with-dynamic-pydantic-fields)
20. ~~SQL-first, Pydantic-second~~
21. ~~Validate file formats~~
22. [Validate url source (if users are able to upload files)](https://github.com/zhanymkanov/fastapi-best-practices#22-validate-url-source-if-users-are-able-to-upload-files)
23. [root_validator to use multiple columns during validation](https://github.com/zhanymkanov/fastapi-best-practices#23-root_validator-to-use-multiple-columns-during-validation)
24. ~~pre=True if data need to be pre-handled before validation~~
25. ~~raise a ValueError in pydantic, if schema faces http client~~
26. ~~remember fastapi response modeling~~
27. if must use sdk, but it's not async, use threadpools
13. [Set tests client async from day 0](https://github.com/zhanymkanov/fastapi-best-practices#14-set-tests-client-async-from-day-0)
14. [BackgroundTasks > asyncio.create_task](https://github.com/zhanymkanov/fastapi-best-practices#16-use-backgroundtasks)
15. [Typing is important](https://github.com/zhanymkanov/fastapi-best-practices#17-typing-is-important)
16. [Save files in chunk](https://github.com/zhanymkanov/fastapi-best-practices#18-dont-hope-your-clients-will-send-small-blobs-save-files-in-chunk)
17. [Be careful with dynamic pydantic fields](https://github.com/zhanymkanov/fastapi-best-practices#19-be-careful-with-dynamic-pydantic-fields)
18. ~~SQL-first, Pydantic-second, Custom-third~~
19. [Validate url source (if users are able to upload files)](https://github.com/zhanymkanov/fastapi-best-practices#22-validate-url-source-if-users-are-able-to-upload-files)
20. [root_validator to use multiple columns during validation](https://github.com/zhanymkanov/fastapi-best-practices#23-root_validator-to-use-multiple-columns-during-validation)
21. ~~pre=True if data need to be pre-handled before validation~~
22. ~~raise a ValueError in pydantic, if schema faces http client~~
23. ~~remember fastapi response modeling~~
24. if must use sdk, but it's not async, use threadpools
25. ~~use linters~~
### 1. Project Structure. Consistent & predictable
There are many ways to structure the project, but the best structure is a structure that is consistent, straightforward and has no surprises.
@@ -42,7 +40,7 @@ There are many ways to structure the project, but the best structure is a struct
- If the frequency and location of the files feels random, then your project structure is bad.
Although, the project structure, where we separate files by their type (e.g. api, crud, models, schemas)
presented by [@tiangolo](https://github.com/tiangolo) is perfect for microservices or projects with fewer scopes,
presented by [@tiangolo](https://github.com/tiangolo) is good for microservices or projects with fewer scopes,
we couldn't fit it into our monolith with a lot of domains and modules.
Structure that I found more scalable and evolvable is inspired by Netflix's [Dispatch](https://github.com/Netflix/dispatch) with some little modifications.
```
@@ -336,7 +334,7 @@ Use /me endpoints for users own resources (e.g. `GET /profiles/me`, `GET /users/
### 7. Don't make your routes async, if you have only blocking I/O operations
Under the hood, FastAPI can [effectively handle](https://fastapi.tiangolo.com/async/#path-operation-functions) both async and sync I/O operations.
- FastAPI calls sync routes in the [threadpool](https://en.wikipedia.org/wiki/Thread_pool)
- FastAPI runs `sync` routes in the [threadpool](https://en.wikipedia.org/wiki/Thread_pool)
and blocking I/O operations won't stop [event loop](https://docs.python.org/3/library/asyncio-eventloop.html)
from executing the tasks.
- Otherwise, if the route is defined as `async` then it's called regularly via `await`
@@ -528,7 +526,8 @@ class AppSettings(BaseSettings):
IS_GOOD_ENV: bool = True
ALLOWED_CORS_ORIGINS: set[AnyUrl]
```
### 11. SQLAlchemy: Set DB keys naming convention from day 0
### 11. SQLAlchemy: Set DB keys naming convention
Explicitly setting the indexes' namings according to your database's convention is preferable over sqlalchemy's.
```python
from sqlalchemy import MetaData
@@ -541,12 +540,31 @@ POSTGRES_INDEXES_NAMING_CONVENTION = {
}
metadata = MetaData(naming_convention=POSTGRES_INDEXES_NAMING_CONVENTION)
```
### 12. Set DB table naming convention immediately from day 0
### 13. Set UUIDs within the app
Setting them in database makes it harder to write integration tests.
### 12. Migrations. Alembic.
1. Migrations must be static and easily revertable.
If your migrations depend on dynamically generated data,
make sure the only thing that changes there is the data itself, not its structure.
2. Generate migrations with descriptive name & slug. Slug is required and should explain the changes.
Set human-readable file template for new migrations.
We use `date` + `slug` pattern, e.g. `2022-08-24_post_content_idx.py`
```
# alembic.ini
file_template = %%(year)d-%%(month).2d-%%(day).2d_%%(slug)s
```
### 13. Set DB tables naming convention
Being consistent with names was important. Some of the rules we followed:
1. lower_case_snake
2. singular form
3. group similar tables with module prefix, e.g. `payment_account`, `payment_bill`, `post`, `post_like`
4. stay consistent across tables, but concrete namings are ok, e.g.
1. use `profile_id` in all tables, but if some of them need only profiles that are creators, use `creator_id`
2. use `post_id` for all abstract tables like `post_like`, `post_view`, but use concrete naming in relevant modules like `course_id` in `chapters.course_id`
5. `_at` suffix for datetime
6. `_date` suffix for date
### 14. Set tests client async from day 0
Writing integration tests with DB will most likely lead to messed up event loop errors in the future.
Set the async test client immediately, e.g. [asyn_asgi_testclient](https://github.com/vinissimus/async-asgi-testclient) or [httpx](https://github.com/encode/starlette/issues/652)
Set the async test client immediately, e.g. [async_asgi_testclient](https://github.com/vinissimus/async-asgi-testclient) or [httpx](https://github.com/encode/starlette/issues/652)
```python
import pytest
from async_asgi_testclient import TestClient
@@ -572,9 +590,11 @@ async def test_create_post(client: TestClient):
assert resp.status_code == 201
```
Unless you have sync db connection (excuse me?) or aren't planning to write integration tests.
### 15. Set postgres identity from day 0
### 16. Use BackgroundTasks
They are stable enough for async (delayed) tasks
### 15. BackgroundTasks > asyncio.create_task
BackgroundTasks can [effectively run](https://github.com/encode/starlette/blob/31164e346b9bd1ce17d968e1301c3bb2c23bb418/starlette/background.py#L25) both blocking and non-blocking I/O operations.
Since the API for sending these tasks will be the same (no need to await coroutines),
it's preferable to use starlette's background tasks.
Don't use it for CPU intensive tasks.
```python
from fastapi import BackgroundTasks
from pydantic import UUID4
@@ -589,7 +609,7 @@ async def send_user_email(worker: BackgroundTasks, user_id: UUID4):
worker.add_task(notifications_service.send_email, user_id) # send email after responding client
return {"status": "ok"}
```
### 17. Typing is important
### 16. Typing is important
FastAPI, Pydantic, and modern IDEs encourage to take use of type hints.
**Without Type Hints**
@@ -600,7 +620,8 @@ FastAPI, Pydantic, and modern IDEs encourage to take use of type hints.
<img src="images/type_hints.png" width="400" height="auto">
### 18. Don't hope your clients will send small BLOBs. Save files in chunk.
### 17. Save files in chunk.
Don't hope your clients will send small files.
```python
import aiofiles
from fastapi import UploadFile
@@ -612,7 +633,7 @@ async def save_video(video_file: UploadFile):
while chunk := await video_file.read(DEFAULT_CHUNK_SIZE):
await f.write(chunk)
```
### 19. Be careful with dynamic pydantic fields
### 18. Be careful with dynamic pydantic fields
If you have a pydantic field that can accept multiple types, be sure validator explicitly knows the difference between those types.
```python
from pydantic import BaseModel
@@ -719,9 +740,8 @@ print(type(p.field_2))
print(type(p.content))
# OUTPUT: Article
```
### 20. SQL-first, Pydantic-second
### 21. Validate file formats
### 22. Validate url source (if users are able to upload files)
### 19. SQL-first, Pydantic-second, Custom-last
### 20. Validate url source (if users are able to upload files and send urls)
Bad users could send strange urls for user facing public objects.
```python
from pydantic import AnyUrl, BaseModel
@@ -744,7 +764,7 @@ class Post(BaseModel):
thumbnail_url: CompanyMediaUrl
```
### 23. root_validator to use multiple columns during validation
### 21. root_validator to use multiple columns during validation
```python
from pydantic import BaseModel, root_validator
@@ -761,11 +781,10 @@ class Profile(BaseModel):
return data
```
### 24. pre if data need to be pre-handled before validation
### 25. you can just raise a ValueError in pydantic schemas, if schemas faces http client
### 22. pre if data need to be pre-handled before validation
### 23. you can just raise a ValueError in pydantic schemas, if schemas faces http client
it wil return a nice response
### 26. don't forget that fastapi converts response Model to Dict then to Model then to JSON
### 24. don't forget that fastapi converts response Model to Dict then to Model then to JSON
it may lead to bugs like model can parse only raw data (e.g. forced data aggregation for raw data)
### 27. if must use sdk, but it's not async, use threadpools
### 28. use linters (black, isort, autoflake)
### 29. set logs from day 0
### 25. if must use sdk, but it's not async, use threadpools
### 26. use linters (black, isort, autoflake)